Recording medium, playback device, and integrated circuit

ABSTRACT

A pair of main-view and sub-view video streams and a graphics stream are recorded on a BD-ROM disc. Metadata is provided in each GOP in the sub-view video stream. The metadata includes offset information. The offset information specifies offset control for a plurality of pictures constituting a GOP. Offset control is to provide a left offset and right offset for the horizontal coordinates in a graphics plane to generate a pair of graphics planes, and to combine them separately with main-view and sub-view video planes. The sub-view video stream is packetized and multiplexed in a transport stream. A header of each TS packet includes a TS priority flag. TS packets containing the metadata have a different value of TS priority flag from TS packets containing sub-view pictures.

This is a continuation of International Application PCT/JP2010/004439,with an international filing date of Jul. 7, 2010.

TECHNICAL FIELD

The present invention relates to a technology for stereoscopic, i.e.three-dimensional (3D), video playback and especially to the structureof stream data on a recording medium.

BACKGROUND ART

In recent years, general interest in 3D video has been increasing. Forexample, amusement park attractions that incorporate 3D video images arepopular. Furthermore, throughout the country, the number of movietheaters showing 3D movies is increasing. Along with this increasedinterest in 3D video, the development of technology that enablesplayback of 3D video images in the home has also been progressing. Thereis demand for this technology to store 3D video content on a portablerecording medium, such as an optical disc, while maintaining the 3Dvideo content at high image quality. Furthermore, there is demand forthe recording medium to be compatible with a two-dimensional (2D)playback device. That is, it is preferable for a 2D playback device tobe able to play back 2D video images and a 3D playback device to be ableto play back 3D video images from the same 3D video content recorded onthe recording medium. Here, a “2D playback device” refers to aconventional playback device that can only play back monoscopic videoimages, i.e. 2D video images, whereas a “3D playback device” refers to aplayback device that can play back 3D video images. Note that in thepresent description, a 3D playback device is assumed to be able to alsoplay back conventional 2D video images.

FIG. 109 is a schematic diagram illustrating the technology for ensuringthe compatibility of an optical disc storing 3D video content with 2Dplayback devices (see, for example, Patent Document 1). An optical discPDS stores two types of video streams. One is a 2D/left-view videostream, and the other is a right-view video stream. A “2D/left-viewvideo stream” represents a 2D video image to be shown to the left eye ofa viewer during 3D playback, i.e. a “left view”. During 2D playback,this stream constitutes the 2D video image. A “right-view video stream”represents a 2D video image to be shown to the right eye of the viewerduring 3D playback, i.e. a “right view”. The left and right-view videostreams have the same frame rate but different presentation timesshifted from each other by half a frame period. For example, when theframe rate of each video stream is 24 frames per second, the frames ofthe 2D/left-view video stream and the right-view video stream arealternately displayed every 1/48 seconds.

As shown in FIG. 109, the left-view and right-view video streams aredivided into a plurality of extents EX1A-C and EX2A-C respectively onthe optical disc PDS. Each extent contains at least one group ofpictures (GOP), GOPs being read together by the optical disc drive.Hereinafter, the extents belonging to the 2D/left-view video stream arereferred to as “2D/left-view extents”, and the extents belonging to theright-view video stream are referred to as “right-view extents”. The2D/left-view extents EX1A-C and the right-view extents EX2A-C arealternately arranged on a track TRC of the optical disc PDS. Each twocontiguous extents EX1A+EX2A, EX1B+EX2B, and EX1C+EX2C have the samelength of playback time. Such an arrangement of extents is referred toas an “interleaved arrangement”. A group of extents recorded in aninterleaved arrangement on a recording medium is used both in 3D videoplayback and 2D video image playback, as described below.

From among the extents recorded on the optical disc PDS, a 2D playbackdevice PL2 causes an optical disc drive DD2 to read only the2D/left-view extents EX1A-C sequentially from the start, skipping thereading of right-view extents EX2A-C. Furthermore, an image decoder VDCsequentially decodes the extents read by the optical disc drive DD2 intoa video frame VFL. In this way, a display device DS2 only displays leftviews, and viewers can watch normal 2D video images.

A 3D playback device PL3 causes an optical disc drive DD3 to alternatelyread 2D/left-view extents and right-view extents from the optical discPDS. When expressed as codes, the extents are read in the order EX1A,EX2A, EX1B, EX2B, EX1C, and EX2C. Furthermore, from among the readextents, those belonging to the 2D/left-view video stream are suppliedto a left-video decoder VDL, whereas those belonging to the right-viewvideo stream are supplied to a right-video decoder VDR. The videodecoders VDL and VDR alternately decode each video stream into videoframes VFL and VFR, respectively. As a result, left views and rightviews are alternately displayed on a display device DS3. Insynchronization with the switching of the views by the display deviceDS3, shutter glasses SHG cause the left and right lenses to becomeopaque alternately. Therefore, a viewer wearing the shutter glasses SHGsees the views displayed by the display device DS3 as 3D video images.

When 3D video content is stored on any recording medium, not only on anoptical disc, the above-described interleaved arrangement of extents isused. The recording medium can thus be used both for playback of 2Dvideo images and 3D video images.

CITATION LIST Patent Literature [Patent Literature 1]

-   Japanese Patent Publication No. 3935507

SUMMARY OF INVENTION Technical Problem

General video content includes, in addition to a video stream, one ormore graphics streams representing graphics images such as subtitles andinteractive screens. When video images are played back from 3D videocontent, the graphics images are also reproduced in three dimensions.Techniques of reproducing them in three dimensions include 2 plane modeand 1 plane+offset mode. 3D video content in 2 plane mode includes apair of graphics streams separately representing left-view graphicsimages and right-view ones. A playback device in 2 plane mode generatesa separate left-view and right-view graphics plane from the graphicsstreams. 3D video content in 1 plane+offset mode includes a graphicsstream representing 2D graphics images, and offset information providedfor the graphics stream. A playback device in 1 plane+offset mode firstgenerates a single graphics plane from the graphics stream, and thenprovides a horizontal offset in the graphics plane in accordance withthe offset information. A pair of left-view and right-view graphicsplanes is thus generated from the graphics stream. In either mode,left-view and right-view graphics images are alternately displayed onthe screen of the display device. As a result, viewers perceive thegraphics images as 3D images.

If the graphics stream and the offset information are contained inseparate files of 3D video content, the playback device in 1plane+offset mode processes these files separately into correspondingdata pieces, and use the data pieces to generate a pair of left-view andright-view graphics images. Note that the graphics images and offsetinformation are generally changed in frame periods. However, reading andanalyzing the file storing the offset information each time a frame isdisplayed has a risk that “the process is not completed in time and theimages cannot be displayed correctly”. Accordingly, in order for theprocess to be synchronized with the frame period without fail, it isnecessary to expand the offset information in the memory in advance. Inthat case, the capacity of the built-in memory in which the file storingthe offset information is to be expanded should necessarily be largebecause the total amount of the offset information per graphics streamis large. Also, when a plurality of graphics images are included in onescene, the built-in memory is required to have even larger capacity. Inthis way, incorporating the graphics stream and the offset informationas separate files into 3D video content prevents a further reduction incapacity of the built-in memory.

In order to solve the above-described problem, the offset information iscontained in the video stream at intervals of GOPs, for example. Thisallows a decoder in a playback device to extract the offset informationfrom the video stream while decoding the video stream. As a result, theplayback device can surely maintain the correspondence between thegraphics stream and the offset information. In addition, the built-inmemory only needs to have a capacity sufficient to expand the offsetinformation per GOP therein, for example. This can easily achieve boththe support of 3D video contents with various graphics streams and thefurther reduction in capacity of the built-in memory.

Here, various means are conceivable as specific means used by thedecoder in a playback device to implement the function to extract offsetinformation from video streams, such as a means for incorporating thefunction into the hardware dedicated to decoding of video streams, and ameans for realizing the function by another hardware or software.However, it is not preferable to vary the data structures of videostreams and offset information among those means.

An object of the present invention is to solve the above problems,particularly to provide a recording medium in which a video stream andoffset information are integrally recorded in a data structure usable incommon in various modes of implementing the function, which is toextract the offset information from the video stream, into a playbackdevice.

Solution to Problem

On a recording medium according to the present invention, a main-viewvideo stream, a sub-view video stream, and a graphics stream arerecorded. The main-view video stream includes main-view picturesconstituting main views of stereoscopic video images. The sub-view videostream includes sub-view pictures and metadata, the sub-view picturesconstituting sub-views of stereoscopic video images. The graphics streamincludes graphics data constituting monoscopic graphics images. Themain-view pictures are each rendered on a main-view video plane, whenbeing played back. The sub-view pictures are each rendered on a sub-viewvideo plane, when being played back. The graphics data is rendered on agraphics plane, when being played back. The metadata is provided in eachgroup of pictures (GOP) constituting the sub-view video stream andincludes offset information. The offset information is controlinformation specifying offset control for a plurality of picturesconstituting a GOP. The offset control is a process to provide a leftoffset and a right offset for horizontal coordinates in the graphicsplane to generate a pair of graphics planes, and then combine the pairof graphics planes separately with the main-view video plane and thesub-view video plane. The sub-view video stream is multiplexed in atransport stream (TS). TS packets constituting the TS each have a headerincluding a TS priority flag that indicates a priority of the TS packet.TS packets containing the metadata have a different value of TS priorityflag from TS packets containing the sub-view pictures.

Advantageous Effects of Invention

The recording medium according to the present invention enables thedecoding unit of a playback device to separate TS packets containing themetadata and TS packets containing the sub-view pictures in accordancewith the values of the TS priority flags.

Accordingly, the decoding unit can be equipped with separate functionunits; one for extracting the offset information from TS packetscontaining the metadata, and the other for decoding TS packetscontaining the sub-view pictures into uncompressed pictures. In thiscase, specific configurations of these function units can be designedindependently of each other. On the other hand, the decoding unit inwhich the function units are integrated enables the integrated functionunit to process all the TS packets containing the sub-view video stream,independently of the values of the TS priority flags. In this way, therecording medium according to the present invention enables a videostream and offset information to be integrally recorded therein in adata structure usable in common in various modes of implementing thefunction, which is to extract the offset information from the videostream, into a playback device.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram showing a home theater system that uses arecording medium according to Embodiment 1 of the present invention.

FIG. 2 is a schematic diagram showing a data structure of a BD-ROM disc101 shown in FIG. 1.

FIGS. 3A and 3B are lists of elementary streams multiplexed in a main TSand a sub-TS on the BD-ROM disc, respectively.

FIG. 4 is a schematic diagram showing the arrangement of TS packets inthe multiplexed stream data 400.

FIG. 5A is a schematic diagram showing a data structure of a TS header501H;

FIG. 5B is a schematic diagram showing the format of a sequence of TSpackets 501 constituting multiplexed stream data; FIG. 5C is a schematicdiagram showing the formation of a sequence of source packets 502composed of the TS packet sequence for multiplexed stream data; and FIG.5D is a schematic diagram of a sector group, in which a sequence ofsource packets 502 are consecutively recorded, in the volume area of theBD-ROM disc.

FIG. 6 is a schematic diagram showing the data structure of the PGstream 600.

FIG. 7 is a schematic diagram showing the pictures for a base-view videostream 701 and a right-view video stream 702 in order of presentationtime.

FIG. 8 is a schematic diagram showing details on a data structure of avideo stream 800.

FIG. 9 is a schematic diagram showing details on a method for storing avideo stream 901 into a PES packet sequence 902.

FIG. 10 is a schematic diagram showing correspondence between PTSs andDTSs assigned to each picture in a base-view video stream 1001 and adependent-view video stream 1002.

FIG. 11 is a schematic diagram showing a data structure of offsetmetadata 1110 included in a dependent-view video stream 1100.

FIG. 12 is a table showing syntax of this offset metadata 1110 shown inFIG. 11.

FIGS. 13A and 13B are schematic diagrams showing offset controls for aPG plane 1310 and IG plane 1320 respectively; and FIG. 13C is aschematic diagram showing 3D graphics images that a viewer 1330 is madeto perceive from 2D graphics images represented by graphics planes shownin FIGS. 13A and 13B.

FIGS. 14A and 14B are graphs showing examples of offset sequences; andFIG. 14C is a schematic diagram showing 3D graphics images reproduced inaccordance with the offset sequences shown in FIGS. 14A and 14B.

FIG. 15 is a schematic diagram showing a PES packet 1510 storing VAU #11500 in the dependent-view video stream, and a sequence of TS packets1520 generated from the PES packet 1510.

FIG. 16 is a schematic diagram showing a sequence of TS packets 1620where TS packets belonging to the first group 1521 and the second group1522 shown in FIG. 15 indicate the same value of TS priority.

FIG. 17A is a schematic diagram showing a data structure of decodingswitch information 1750; and FIGS. 17B and 17C are schematic diagramsshowing sequences of decoding counters 1710, 1720, 1730, and 1740allocated to each picture in a base-view video stream 1701 and adependent-view video stream 1702.

FIG. 18 is a schematic diagram showing a data structure of a PMT 1810.

FIG. 19 is a schematic diagram showing a physical arrangement ofmultiplexed stream data on the BD-ROM disc.

FIG. 20A is a schematic diagram showing the arrangement of the main TS2001 and sub-TS 2002 recorded separately and consecutively on a BD-ROMdisc; FIG. 20B is a schematic diagram showing an arrangement ofdependent-view data blocks D[0], D[1], D[2], . . . and base-view datablocks B[0], B[1], B[2], . . . recorded alternately on the BD-ROM disc101 according to Embodiment 1 of the present invention; and FIGS. 20Cand 20D are schematic diagrams showing examples of the extent ATC timesfor a dependent-view data block group D[n] and a base-view data blockgroup B[n] recorded in an interleaved arrangement (n=0, 1, 2).

FIG. 21 is a schematic diagram showing a playback path 2101, 2102 in 2Dplayback mode and L/R mode for an extent block group 1901-1903.

FIG. 22 is a schematic diagram showing a data structure of 2D clipinformation file (01000.clpi) 231.

FIG. 23A is a schematic diagram showing a data structure of an entry map2230; FIG. 23B is a schematic diagram showing source packets in a sourcepacket group 2310 belonging to a file 2D 241 that are associated witheach EP_ID 2305 by the entry map 2230; and FIG. 23C is a schematicdiagram showing a data block group D[n], B[n] (n=0, 1, 2, 3, . . . ) ona BD-ROM disc 101 corresponding to the source packet group 2310.

FIG. 24A is a schematic diagram showing a data structure of extent startpoints 2242; FIG. 24B is a schematic diagram showing a data structure ofextent start points 2420 included in dependent-view clip informationfile (02000.clpi) 232; FIG. 24C is a schematic diagram representing thebase-view data blocks B[0], B[1], B[2], . . . extracted from the file SS244A by the playback device 102 in 3D playback mode; FIG. 24D is aschematic diagram representing correspondence between dependent-viewextents EXT2[0], EXT2[1], . . . belonging to the file DEP (02000.m2ts)242 and the SPNs 2422 shown by the extent start points 2420; and FIG.24E is a schematic diagram showing correspondence between an extent SSEXTSS[0] belonging to the file SS 244A and an extent block on the BD-ROMdisc.

FIG. 25 is a schematic diagram showing correspondence between a singleextent block 2500 recorded on the BD-ROM disc and each of the extentblock groups in a file 2D 2510, file base 2511, file DEP 2512, and fileSS 2520.

FIG. 26 is a schematic diagram showing an example of entry points set ina base-view video stream 2610 and a dependent-view video stream 2620.

FIG. 27 is a schematic diagram showing a data structure of a 2D playlistfile.

FIG. 28 is a schematic diagram showing a data structure of PI #N shownin FIG. 27.

FIGS. 29A and 29B are schematic diagrams showing a correspondencebetween two playback sections 2901 and 2902 to be connected when CC is“5” or “6”.

FIG. 30 is a schematic diagram showing a correspondence between PTSsindicated by a 2D playlist file (00001.mpls) 221 and sections playedback from a file 2D (01000.m2ts) 241.

FIG. 31 is a schematic diagram showing a data structure of a 3D playlistfile.

FIG. 32 is a schematic diagram showing an STN table 3205 included in amain path 3101 of the 3D playlist file shown in FIG. 31.

FIG. 33 is a schematic diagram showing a data structure of the STN tableSS 3130 shown in FIG. 31.

FIG. 34 is a schematic diagram showing correspondence between PTSsindicated by a 3D playlist file (00002.mpls) 222 and sections playedback from a file SS (01000.ssif) 244A.

FIG. 35 is a schematic diagram showing a data structure of an index file(index.bdmv) 211 shown in FIG. 2.

FIG. 36 is a flowchart of processing whereby the playback device 102shown in FIG. 1 selects a playlist file for playback by using six typesof determination processes.

FIG. 37 is a functional block diagram of a 2D playback device 3700.

FIG. 38 is a list of system parameters (SPRMs) stored in the playervariable storage unit 3736 shown in FIG. 37.

FIG. 39 is a flowchart of 2D playlist playback processing by a playbackcontrol unit 3735 shown in FIG. 37.

FIG. 40 is a functional block diagram of the system target decoder 3725shown in FIG. 37.

FIG. 41A is a flowchart of processing whereby the PG decoder 4072 shownin FIG. 40 decodes a graphics object from one data entry in the PGstream; and FIGS. 41B-41E are schematic diagrams showing the graphicsobject changing as the processing proceeds.

FIG. 42 is a functional block diagram of a 3D playback device 4200.

FIG. 43 is a table showing a data structure of SPRM(27) and SPRM(28)stored in the player variable storage unit 4236 shown in FIG. 42.

FIG. 44 is a flowchart of 3D playlist playback processing by a playbackcontrol unit 4235 shown in FIG. 42.

FIG. 45 is a functional block diagram of the system target decoder 4225shown in FIG. 42 that implements the function to extract offset metadataby using a first means.

FIG. 46 is a functional block diagram of a system of processing videostreams in the system target decoder 4225 shown in FIG. 42 thatimplements the function to extract offset metadata by using a secondmeans.

FIG. 47 is a functional block diagram of a plane adder 4226 shown inFIG. 42.

FIG. 48 is a flowchart of offset control by the cropping units 4731-4734shown in FIG. 47.

FIG. 49B is a schematic diagram showing PG plane data GP to which thesecond cropping unit 4732 is to provide offset control; and FIGS. 49Aand 49C are a schematic diagrams showing PG plane data RPG to which aright offset has been provided and PG plane data LPG to which a leftoffset has been provided.

FIG. 50 is a schematic diagram showing a PES packet 5010 storing VAU #15000 in the dependent-view video stream and a sequence of TS packets5020 generated from the PES packet 5010.

FIG. 51 is a functional block diagram showing a system of processingvideo streams in the system target decoder 5125 that extracts offsetmetadata from the TS packet sequence 5020 shown in FIG. 50.

FIG. 52A is a schematic diagram showing a data structure of offsetmetadata 5200 that uses a completion function; FIG. 52B is a graphshowing the types of elements in the completion function; and FIG. 52Cis a graph showing offset values calculated by a 3D playback device fromoffset sequence IDs=0, 1, 2 shown in FIG. 52A.

FIG. 53 is a schematic diagram showing (i) a data structure of a 3Dplaylist file 5300 that includes a plurality of sub-paths and (ii) adata structure of a file 2D 5310 and two files DEP 5321 and 5322 thatare referred to by the 3D playlist file 5300.

FIG. 54 is a schematic diagram showing an STN table 5400 in which two ormore offset adjustment values are set for one piece of stream data.

FIGS. 55A-55C are schematic diagrams showing parallaxes PRA, PRB, andPRC between left and right views displayed on a 32-inch screen SCA,50-inch screen SCB, and 100-inch screen SCC, respectively.

FIG. 56A is a schematic diagram showing a correspondence table betweenscreen sizes and output offset adjustment values; and FIG. 56B is agraph representing a function between screen sizes and output offsetadjustment values.

FIG. 57 is a block diagram showing the components of a 3D playbackdevice required for output offset adjustment.

FIG. 58A is a schematic diagram showing a data structure of adependent-view video stream 5800 representing only still images; andFIG. 58B is a schematic diagram showing a left-view video plane sequence5821, a right-view video plane sequence 5822, and a graphics planesequence 5830 that are played back in accordance with such a 3Dplaylist.

FIG. 59 is a block diagram of the display device 103 that performs theprocessing for compensating a misalignment between left and right views.

FIG. 60A is a plan view schematically showing horizontal angles of viewHAL and HAR for a pair of video cameras CML and CMR filming 3D videoimages; FIGS. 60B and 60C are schematic diagrams showing a left view LVfilmed by the left-video camera CML and a right view RV captured by theright-video camera CMR, respectively; and FIGS. 60D and 60E areschematic diagrams respectively showing a left view LV represented bythe processed left-video plane and a right view RV represented by theprocessed right-video plane.

FIG. 61A is a plan view schematically showing vertical angles of viewVAL and VAR for a pair of video cameras CML and CMR filming 3D videoimages; FIG. 61B is a schematic diagram showing a left view LV filmed bythe left-video camera CML and a right view RV captured by theright-video camera CMR; and FIG. 61C is a schematic diagram showing aleft view LV represented by the processed left-video plane and a rightview RV represented by the processed right-video plane.

FIG. 62A is a schematic diagram showing an example of graphics imagesrepresented by a graphics plane GPL; FIGS. 62B and 62C are schematicdiagrams respectively showing processes of providing a right and leftoffset to the graphics plane GPL; and FIGS. 62D and 62E are schematicdiagrams showing graphics images represented by the graphics planes GP1and GP2 with the right and left offsets, respectively.

FIG. 63 is a schematic diagram showing a condition regarding thearrangement of graphic elements for graphics planes played back from aPG stream or an IG stream on a BD-ROM disc and for a graphics planegenerated by a playback device 102.

FIGS. 64A1 and 64A2 are schematic diagrams showing the same screen inthe letterbox display; FIGS. 64B and 64C are schematic diagrams showingthe screens in which the primary video plane has been provided withupward and downward offsets of 131 pixels, respectively; and FIG. 64D isa schematic diagram showing the screen in which the primary video planehas been provided with an upward offset of 51 pixels.

FIG. 65 is a functional block diagram showing the structure of theplayback device required for the video shift.

FIG. 66A is a table showing the data structures of the SPRM(32) andSPRM(33); and FIG. 66B is a schematic diagram showing the STN table inthe playlist file for the video content of the letterbox display.

FIGS. 67A-67C are schematic diagrams showing primary video planes VPA,VPB, and VPC processed by the video shift unit 6501 in the Up mode, Keepmode, and Down mode, respectively; FIGS. 67D-67F are schematic diagramsshowing PG planes PGD, PGE, and PGF processed by the second croppingunit 4632 in the Up mode, Keep mode, and Down mode, respectively; andFIGS. 67G-67I are schematic diagrams showing plane data PLG, PLH, andPLI combined by the second adder 4642 in the Up mode, Keep mode, andDown mode, respectively.

FIG. 68A is a schematic diagram showing another example of the STN tablein the playlist file for the video content of the letterbox display; andFIG. 68B is a schematic diagram showing the order of registration of aplurality of pieces of stream attribute information 6803, each includingthe video shift mode 6812, in the STN table shown in FIG. 68A.

FIG. 69 is a functional block diagram showing another example of thestructure of the playback device required for the video shift.

FIG. 70A is a schematic diagram showing the data structure of theSPRM(37) in the player variable storage unit 4236; FIG. 70B is aschematic diagram showing the video image IMG and subtitle SUB displayedon the screen SCR in the case where the background color of the subtitlerepresented by the PG stream is set to the colorless transparent; andFIG. 70C is a schematic diagram showing the video image IMG and subtitleSUB displayed on the screen SCR in the case where a color coordinatevalue of the background color of the subtitle is stored in the SPRM(37).

FIG. 71A is a schematic diagram showing a further another example of theSTN table in the playlist file for the video content of the letterboxdisplay; and FIG. 71B is a functional block diagram showing a furtheranother example of the structure of the playback device required for thevideo shift.

FIG. 72A is a schematic diagram showing the subtitles SB1 and SB2 thatcorrespond to the Keep mode; FIG. 72B is a schematic diagram showing thesubtitles SB1 and SB2 that correspond to the Down mode; FIG. 72C is aschematic diagram showing the subtitle SB1 displayed in the Keep mode;and FIG. 72D is a schematic diagram showing the subtitle SB3 displayedin the Up mode when the video upward subtitle 7110 is not registered inthe STN table.

FIGS. 73A and 73B are lists of elementary streams multiplexed in a firstsub-TS and a second sub-TS on a BD-ROM disc, respectively.

FIG. 74 is a schematic diagram showing a data structure of the STN tableSS 3130 according to Embodiment 2 of the present invention.

FIG. 75 is a functional block diagram of a system target decoder 7525according to Embodiment 2 of the present invention.

FIG. 76 is a partial functional block diagram of the plane adder 7526 in2 plane mode.

FIGS. 77A, 77B, and 77C are schematic diagrams showing a left-viewgraphics image GOB0 represented by the 2D PG stream and a right-viewgraphics images GOB1-GOB3 represented by the right-view PG stream; andFIGS. 77D, 77E, and 77F are schematic diagrams showing the offsetcontrol performed onto the left-view graphics image shown in FIGS. 77A,77B, and 77C.

FIG. 78 is a functional block diagram of a recording device 7800according to Embodiment 3 of the present invention.

FIGS. 79A and 79B are schematic diagrams respectively showing a picturein a left view and a right view used to display one scene of 3D videoimages; and FIG. 79C is a schematic diagram showing depth informationcalculated from these pictures by the video encoder 7802.

FIG. 80 is a flowchart of a method for recording movie content on aBD-ROM disc using the recording device 7800 shown in FIG. 78.

FIG. 81 is a schematic diagram showing a method to align extent ATCtimes between consecutive data blocks.

FIGS. 82A-82C are schematic diagrams illustrating the principle behindplayback of 3D video images (stereoscopic video images) in a methodusing parallax video images.

FIG. 83 is a schematic diagram showing an example of constructing aleft-view LVW and a right-view RVW from the combination of a 2D videoimage MVW and a depth map DPH.

FIG. 84 is a block diagram showing playback processing in the playbackdevice in 2D playback mode.

FIG. 85A is a graph showing the change in the data amount DA stored inthe read buffer 3721 during operation of the playback processing shownin FIG. 84 in 2D playback mode; and FIG. 85B is a schematic diagramshowing the correspondence between an extent block 8510 for playback anda playback path 8520 in 2D playback mode.

FIG. 86 is an example of a correspondence table between jump distancesS_(JUMP) and maximum jump times T_(JUMP) _(—) _(MAX) for a BD-ROM disc.

FIG. 87 is a block diagram showing playback processing in the playbackdevice in 3D playback mode.

FIGS. 88A and 88B are graphs showing changes in data amounts DA1 and DA2stored in RB1 4221 and RB2 4222 shown in FIG. 87 when 3D video imagesare played back seamlessly from a single extent block; and FIG. 88C is aschematic diagram showing a correspondence between the extent block 8810and a playback path 8820 in 3D playback mode.

FIG. 89B is a schematic diagram showing an (M+1)^(th) (the letter Mrepresents an integer greater than or equal to 1) extent block 8901 and(M+2)^(th) extent block 8902 and the correspondence between these extentblocks 8901 and 8902 and a playback path 8920 in 3D playback mode; andFIG. 89A is a graph group showing changes in data amounts DA1 and DA2stored in RB1 4221 and RB2 4222, as well as the changes in the sumDA1+DA2, when 3D video images are continually played back seamlesslyfrom two extent blocks 8901 and 8902.

FIGS. 90A and 90B are graphs showing changes in data amounts DA1 and DA2stored in RB1 4221 and RB2 4222 when 3D video images are played backseamlessly from the two consecutive extent blocks 8901 and 8902 shown inFIG. 89B.

FIG. 91 is a block diagram showing the video stream processing systemprovided in the system target decoder 4225 in the 3D playback mode.

FIGS. 92A and 92B are graphs showing temporal changes in the base-viewtransfer rate R_(EXT1) and the dependent-view transfer rate R_(EXT2) inthat case, respectively; and FIG. 92C is a graph showing the temporalchange in the sum of the base-view transfer rate R_(EXT1) and thedependent-view transfer rate R_(EXT2) shown in FIGS. 92A and 92B.

FIG. 93 is a schematic diagram showing the relationships between the TSpackets that are transferred in the system target decoder from thesource depacketizer to the PID filter and the ATC times.

FIG. 94A is a table showing the maximum extent sizes maxS_(EXT1)[n] andmaxS_(EXT2)[n] for an extent pair at various combinations of thebase-view transfer rate R_(EXT1)[n] and the dependent-view transfer rateR_(EXT2)[n]; FIG. 94B is a schematic diagram showing that an extent pairEXT1[n], EXT2[n] is located at the top of an extent block 9401 arrangedafter a layer boundary LB, and the base-view data block B[n] of theextent pair is arranged before the dependent-view data block D[n]thereof.

FIGS. 95A and 95B are graphs showing changes in amounts DA1, DA2 of datastored in RB1 and RB2, respectively, when 3D video images are playedback seamlessly from the two extent blocks 9401 and 9402 shown in FIG.94B.

FIG. 96A is a schematic diagram showing the syntax of the extent startpoint in the case where the order of data blocks is reversed in theextent pair located in the middle of the extent block; FIG. 96B is aschematic diagram showing the relationships between the base-view extentEXT1[k] (k=0, 1, 2, . . . ) belonging to the file base and the extentstart flag indicated by the extent start point; FIG. 96C is a schematicdiagram showing the relationships between the dependent-view extentEXT2[k] belonging to the file DEP and the extent start flag; and FIG.96D is a schematic diagram showing the relationships between the extentSS EXTSS[0] belonging to the file SS and the extent blocks on the BD-ROMdisc.

FIG. 97C is a schematic diagram showing an arrangement of a data blockwhich requires the largest capacity of RB1 4221; FIGS. 97A and 97B aregraphs showing changes in amounts DA1, DA2 of data stored in RB1 4221and RB2 4222, respectively, when 3D video images are played backseamlessly from the two extent blocks 9701 and 9702 shown in FIG. 97C;FIG. 97F is a schematic diagram showing an arrangement of a data blockwhich requires the largest capacity of RB2 4222; and FIGS. 97D and 97Eare graphs showing changes in amounts DA1, DA2 of data stored in RB14221 and RB2 4222, respectively, when 3D video images are played backseamlessly from the two extent blocks 9703 and 9704 shown in FIG. 97F.

FIG. 98C is a schematic diagram showing an extent block 9810 whichincludes in the middle thereof an extent pair in which the order of datablocks is reversed; and

FIGS. 98A and 98B are graphs showing changes in amounts DA1, DA2 of datastored in RB1 4221 and RB2 4222, respectively, when 3D video images areplayed back seamlessly from the extent block 9801 shown in FIG. 98C.

FIG. 99 is a schematic diagram showing a relationship between an extentblock 9900 and AV stream files 9910-9920, the extent block 9900including an extent pair in the middle thereof, the extent pair havingdata blocks in reverse order.

FIG. 100 is a schematic diagram showing arrangement 1 of a data blockgroup recorded before and after a layer boundary LB on a BD-ROM disc.

FIG. 101 is a schematic diagram showing a playback path A110 in 2Dplayback mode and a playback path A120 in 3D playback mode for the datablock group in arrangement 1 shown in FIG. 100.

FIG. 102 is a schematic diagram showing arrangement 2 of a data blockgroup recorded before and after a layer boundary LB on a BD-ROM disc.

FIG. 103 is a schematic diagram showing a playback path A310 in 2Dplayback mode and a playback path A320 in 3D playback mode for the datablock group in arrangement 2 shown in FIG. 102.

FIG. 104 is a schematic diagram showing the relationships between theread time S_(EXT1)[3]/R_(UD72) of the block exclusively for 3D playbackB[3]_(SS) located at the end of the second extent block A202 and thelower limit of the capacity of RB2 4222.

FIG. 105 is a schematic diagram showing entry points A510 and A520 setfor extents EXT1[k] and EXT2[k] (the letter k represents an integergreater than or equal to 0) in a file base A501 and a file DEP A502.

FIG. 106A is a schematic diagram showing a playback path when extent ATCtimes and playback times of the video stream differ between contiguousbase-view data blocks and dependent-view data blocks; and FIG. 106B is aschematic diagram showing a playback path when the playback times of thevideo stream are equal for contiguous base-view and dependent-view datablocks.

FIG. 107A is a schematic diagram showing a playback path for multiplexedstream data supporting multi-angle; FIG. 107B is a schematic diagramshowing a data block group A701 recorded on a BD-ROM disc and acorresponding playback path A702 in L/R mode; and FIG. 107C is aschematic diagram showing an extent block formed by stream data Ak, Bk,and Ck for different angles.

FIG. 108 is a schematic diagram showing (i) a data block group A801constituting a multi-angle period and (ii) a playback path A810 in 2Dplayback mode and playback path A820 in L/R mode that correspond to thedata block group A801.

FIG. 109 is a schematic diagram illustrating the technology for ensuringthe compatibility of an optical disc storing 3D video content with 2Dplayback devices.

FIG. 110 is a functional block diagram of a playback device realized byusing the integrated circuit 3 according to Embodiment 4 of the presentinvention.

FIG. 111 is a functional block diagram showing a representativestructure of the stream processing unit 5 shown in FIG. 110.

FIG. 112 is a functional block diagram of the switching unit 53 and thesurrounding units shown in FIG. 110 when the switching unit 53 is DMAC.

FIG. 113 is a functional block diagram showing a representativestructure of the AV output unit 8 shown in FIG. 110.

FIG. 114 is a schematic diagram showing one example of the method ofusing the memory 2 during the process of superimposing images.

FIG. 115 is a schematic diagram showing a method of superimposing thegraphics plane on the left-view plane by using the memory 2 shown inFIG. 114.

FIG. 116 is a schematic diagram showing a method of superimposing thegraphics plane on the right-view plane by using the memory 2 shown inFIG. 114.

FIG. 117 is a schematic diagram showing another example of the method ofusing the memory 2 during the process of superimposing images.

FIG. 118 is a detailed functional block diagram of the AV output unit 8and the data output unit in the playback device shown in FIG. 113.

FIGS. 119A and 119B are schematic diagrams showing examples of thetopology of a control bus and a data bus arranged in the integratedcircuit 3 shown in FIG. 110.

FIG. 120 is a functional block diagram showing the structure of theintegrated circuit according to Embodiment 4 and the surrounding units,which are incorporated in a display device.

FIG. 121 is a detailed functional block diagram of the AV output unit 8shown in FIG. 120.

FIG. 122 is a flowchart of playback processing by a playback deviceusing the integrated circuit 3 shown in FIG. 110.

FIG. 123 is a flowchart showing details of the steps S1-S6 shown in FIG.122.

DESCRIPTION OF EMBODIMENTS

The following describes a recording medium and a playback devicepertaining to preferred Embodiments of the present invention withreference to the drawings.

Embodiment 1

FIG. 1 is a schematic diagram showing a home theater system that uses arecording medium according to Embodiment 1 of the present invention.This home theater system adopts a 3D video image (stereoscopic videoimage) playback method that uses parallax video images, and inparticular adopts an alternate-frame sequencing method as a displaymethod (see <<Supplementary Explanation>> for details). As shown in FIG.1, this home theater system plays back a recording medium 101 andincludes a playback device 102, a display device 103, shutter glasses104, and a remote control 105. The playback device 102 and the displaydevice 103 are provided independently of each other as shown in FIG. 1.Alternatively, the playback device 102 and the display device 103 may beprovided as one unit.

The recording medium 101 is a read-only Blu-ray disc (BD)™, i.e. aBD-ROM disc. The recording medium 101 can be a different portablerecording medium, such as an optical disc with a different format suchas DVD or the like, a removable hard disk drive (HDD), or asemiconductor memory device such as an SD memory card. This recordingmedium, i.e. the BD-ROM disc 101, stores movie content as 3D videoimages. This content includes video streams representing a left view anda right view for the 3D video images. The content may further include avideo stream representing a depth map for the 3D video images. Thesevideo streams are arranged on the BD-ROM disc 101 in units of datablocks and are accessed using a file structure described below. Thevideo streams representing the left view or the right view are used byboth a 2D playback device and a 3D playback device to play the contentback as 2D video images. Conversely, a pair of video streamsrepresenting a left view and a right view, or a pair of video streamsrepresenting either a left view or a right view and a depth map, areused by a 3D playback device to play the content back as 3D videoimages.

A BD-ROM drive 121 is mounted on the playback device 102. The BD-ROMdrive 121 is an optical disc drive conforming to the BD-ROM format. Theplayback device 102 uses the BD-ROM drive 121 to read content from theBD-ROM disc 101. The playback device 102 further decodes the contentinto video data/audio data. The playback device 102 is a 3D playbackdevice and can play the content back as both 2D video images and as 3Dvideo images. Hereinafter, the operational modes of the playback device102 when playing back 2D video images and 3D video images arerespectively referred to as “2D playback mode” and “3D playback mode”.In 2D playback mode, video data only includes either a left-view or aright-view video frame. In 3D playback mode, video data includes bothleft-view and right-view video frames.

3D playback mode is further divided into left/right (L/R) mode and depthmode. In “L/R mode”, a pair of left-view and right-view video frames isgenerated from a combination of video streams representing the left viewand right view. In “depth mode”, a pair of left-view and right-viewvideo frames is generated from a combination of video streamsrepresenting either a left view or a right view and a depth map. Theplayback device 102 is provided with an L/R mode. The playback device102 may be further provided with a depth mode.

The playback device 102 is connected to the display device 103 via aHigh-Definition Multimedia Interface (HDMI) cable 122. The playbackdevice 102 converts video data/audio data into a video signal/audiosignal in the HDMI format, and transmits the signals to the displaydevice 103 via the HDMI cable 122. In 2D playback mode, only one ofeither the left-view or the right-view video frame is multiplexed in thevideo signal. In 3D playback mode, both the left-view and the right-viewvideo frames are time-multiplexed in the video signal. Additionally, theplayback device 102 exchanges CEC messages with the display device 103via the HDMI cable 122. The playback device 102 can thus ask the displaydevice 103 whether it supports playback of 3D video images.

The display device 103 is a liquid crystal display. Alternatively, thedisplay device 103 can be another type of flat panel display, such as aplasma display, an organic EL display, etc., or a projector. The displaydevice 103 displays video on the screen 131 in response to a videosignal, and causes the speakers to produce audio in response to an audiosignal. The display device 103 supports playback of 3D video images.During playback of 2D video images, either the left view or the rightview is displayed on the screen 131. During playback of 3D video images,the left view and right view are alternately displayed on the screen131.

The display device 103 includes a left/right signal transmitting unit132. The left/right signal transmitting unit 132 transmits a left/rightsignal LR to the shutter glasses 104 via infrared rays or by radiotransmission. The left/right signal LR indicates whether the imagecurrently displayed on the screen 131 is a left-view or a right-viewimage. During playback of 3D video images, the display device 103detects switching of frames by distinguishing between a left-view frameand a right-view frame based on a control signal that accompanies avideo signal. Furthermore, the display device 103 causes the left/rightsignal transmitting unit 132 to switch the left/right signal LRsynchronously with the detected switching of frames.

The shutter glasses 104 include two liquid crystal display panels 141Land 141R and a left/right signal receiving unit 142. The liquid crystaldisplay panels 141L and 141R respectively constitute the left and rightlens parts. The left/right signal receiving unit 142 receives aleft/right signal LR, and in accordance with changes therein, transmitsthe signal to the left and right liquid crystal display panels 141L and141R. In response to the signal, each of the liquid crystal displaypanels 141L and 141R either lets light pass through the entire panel orshuts light out. For example, when the left/right signal LR indicates aleft-view display, the liquid crystal display panel 141L for the lefteye lets light pass through, while the liquid crystal display panel 141Rfor the right eye shuts light out. When the left/right signal LRindicates a right-view display, the display panels act oppositely. Thetwo liquid crystal display panels 141L and 141R thus alternately letlight pass through in sync with the switching of frames. As a result,when the viewer looks at the screen 131 while wearing the shutterglasses 104, the left view is shown only to the viewer's left eye, andthe right view is shown only to the right eye. The viewer is made toperceive the difference between the images seen by each eye as thebinocular parallax for the same stereoscopic image, and thus the videoimage appears to be stereoscopic.

The remote control 105 includes an operation unit and a transmittingunit. The operation unit includes a plurality of buttons. The buttonscorrespond to each of the functions of the playback device 102 and thedisplay device 103, such as turning the power on or off, starting orstopping playback of the BD-ROM disc 101, etc. The operation unitdetects when the user presses a button and conveys identificationinformation for the button to the transmitting unit as a signal. Thetransmitting unit converts this signal into a signal IR and outputs itvia infrared rays or radio transmission to the playback device 102 orthe display device 103. On the other hand, the playback device 102 anddisplay device 103 each receive this signal IR, determine the buttonindicated by this signal IR, and execute the function associated withthe button. In this way, the user can remotely control the playbackdevice 102 or the display device 103.

<Data Structure of the BD-ROM Disc>

FIG. 2 is a schematic diagram showing a data structure of a BD-ROM disc101. As shown in FIG. 2, a Burst Cutting Area (BCA) 201 is provided atthe innermost part of the data recording area on the BD-ROM disc 101.Only the BD-ROM drive 121 is permitted to access the BCA, and access byapplication programs is prohibited. The BCA 201 can thus be used astechnology for copyright protection. In the data recording area outsideof the BCA 201, tracks spiral from the inner to the outer circumference.In FIG. 2, a track 202 is schematically extended in a transversedirection. The left side represents the inner circumferential part ofthe disc 101, and the right side represents the outer circumferentialpart. As shown in FIG. 2, track 202 contains a lead-in area 202A, avolume area 202B, and a lead-out area 202C in order from the innercircumference. The lead-in area 202A is provided immediately on theoutside edge of the BCA 201. The lead-in area 202A includes informationnecessary for the BD-ROM drive 121 to access the volume area 202B, suchas the size, the physical address, etc. of the data recorded in thevolume area 202B. The lead-out area 202C is provided on the outermostcircumferential part of the data recording area and indicates the end ofthe volume area 202B. The volume area 202B includes application datasuch as video images, audio, etc.

The volume area 202B is divided into small areas 202D called “sectors”.The sectors have a common size, for example 2048 bytes. Each sector 202Dis consecutively assigned a serial number in order from the top of thevolume area 202B. These serial numbers are called logical block numbers(LBN) and are used in logical addresses on the BD-ROM disc 101. Duringreading of data from the BD-ROM disc 101, data to be read is specifiedthrough designation of the LBN for the destination sector. The volumearea 202B can thus be accessed in units of sectors. Furthermore, on theBD-ROM disc 101, logical addresses are substantially the same asphysical addresses. In particular, in an area where the LBNs areconsecutive, the physical addresses are also substantially consecutive.Accordingly, the BD-ROM drive 121 can consecutively read data fromsectors having consecutive LBNs without making the optical pickupperform a seek.

The data recorded in the volume area 202B is managed under apredetermined file system. Universal Disc Format (UDF) is adopted asthis file system. Alternatively, the file system may be ISO9660. Thedata recorded on the volume area 202B is represented in a directory/fileformat in accordance with the file system (see the <<SupplementaryExplanation>> for details). In other words, the data is accessible inunits of directories or files.

<<Directory/File Structure on the BD-ROM Disc>>

FIG. 2 further shows the directory/file structure of the data stored inthe volume area 202B on a BD-ROM disc 101. As shown in FIG. 2, in thisdirectory/file structure, a BD movie (BDMV) directory 210 is locateddirectly below a ROOT directory 203. Below the BDMV directory 210 are anindex file (index.bdmv) 211 and a movie object file (MovieObject.bdmv)212.

The index file 211 contains information for managing as a whole thecontent recorded on the BD-ROM disc 101. In particular, this informationincludes both information to make the playback device 102 recognize thecontent, as well as an index table. The index table is a correspondencetable between a title constituting the content and a program to controlthe operation of the playback device 102. This program is called an“object”. Object types are a movie object and a BD-J (BD Java™) object.

The movie object file 212 generally stores a plurality of movie objects.Each movie object includes a sequence of navigation commands. Anavigation command is a control command causing the playback device 102to execute playback processes similar to general DVD players. Types ofnavigation commands are, for example, a read-out command to read out aplaylist file corresponding to a title, a playback command to play backstream data from an AV stream file indicated by a playlist file, and atransition command to make a transition to another title. Navigationcommands are written in an interpreted language and are deciphered by aninterpreter, i.e. a job control program, included in the playback device102, thus making the control unit execute the desired job. A navigationcommand is composed of an opcode and an operand. The opcode describesthe type of operation that the playback device 102 is to execute, suchas dividing, playing back, or calculating a title, etc. The operandindicates identification information targeted by the operation such asthe title's number, etc. The control unit of the playback device 102calls a movie object in response, for example, to a user operation andexecutes navigation commands included in the called movie object in theorder of the sequence. In a manner similar to general DVD players, theplayback device 102 first displays a menu on the display device 103 toallow the user to select a command. The playback device 102 thenexecutes playback start/stop of a title, switches to another title, etc.in response to the selected command, thereby dynamically changing theprogress of video playback.

As shown in FIG. 2, the BDMV directory 210 further contains a playlist(PLAYLIST) directory 220, a clip information (CLIPINF) directory 230, astream (STREAM) directory 240, a BD-J object (BDJO: BD Java Object)directory 250, and a Java archive (JAR: Java Archive) directory 260.

Three types of AV stream files, (01000.m2ts) 241, (02000.m2ts) 242, and(03000.m2ts) 243, as well as a stereoscopic interleaved file (SSIF)directory 244 are located directly under the STREAM directory 240. Twotypes of AV stream files, (01000.ssif) 244A and (02000.ssif) 244B arelocated directly under the SSIF directory 244.

An “AV stream file” refers to a file, from among an actual video contentrecorded on a BD-ROM disc 101, that complies with the file formatdetermined by the file system. Such an actual video content generallyrefers to stream data in which different types of stream datarepresenting video, audio, subtitles, etc., i.e. elementary streams,have been multiplexed. The multiplexed stream data can be broadlydivided into two types: a main transport stream (TS), and a sub-TS. A“main TS” is multiplexed stream data that includes a base-view videostream as a primary video stream. A “base-view video stream” is a videostream that can be played back independently and that represents 2Dvideo images. These 2D video images are referred to as the “base view”or the “main view”. A “sub-TS” is multiplexed stream data that includesa dependent-view video stream as a primary video stream. A“dependent-view video stream” is a video stream that requires abase-view video stream for playback and represents 3D video images bybeing combined with the base-view video stream. The types ofdependent-view video streams are a right-view video stream, left-viewvideo stream, and depth map stream. When the base view is the left viewof 3D video images, a “right-view video stream” is a video streamrepresenting the right view of the 3D video images. The reverse is truefor a “left-view video stream”. When the base view is a projection of 3Dvideo images on a virtual 2D screen, a “depth map stream” is stream datarepresenting a depth map for the 3D video images. The 2D video images ordepth map represented by the dependent-view video stream are referred toas a “dependent view” or “sub-view”.

Depending on the type of multiplexed stream data stored therein, AVstream files are divided into three types: file 2D, file dependent(hereinafter, abbreviated as “file DEP”), and interleaved file(hereinafter, abbreviated as “file SS”). A “file 2D” is an AV streamfile for playback of 2D video images in 2D playback mode and includes amain TS. A “file DEP” is an AV stream file that includes a sub-TS. A“file SS” is an AV stream file that includes a main TS and a sub-TSrepresenting the same 3D video images. In particular, a file SS sharesits main TS with a certain file 2D and shares its sub-TS with a certainfile DEP. In other words, in the file system on the BD-ROM disc 101, amain TS can be accessed by both a file SS and a file 2D, and a sub TScan be accessed by both a file SS and a file DEP. This setup, whereby asequence of data recorded on the BD-ROM disc 101 is common to differentfiles and can be accessed by all of the files, is referred to as “filecross-link”.

In the example shown in FIG. 2, the first AV stream file (01000.m2ts)241 is a file 2D, and the second AV stream file (02000.m2ts) 242 and thethird AV stream file (03000.m2ts) 243 are both a file DEP. In this way,files 2D and files DEP are located directly below the STREAM directory240. The first AV stream file, i.e. the base-view video stream thatincludes the file 2D 241, represents a left view of 3D video images. Thesecond AV stream file, i.e. the dependent-view video stream thatincludes the first file DEP 242, includes a right-view video stream. Thethird AV stream file, i.e. the dependent-view video stream that includesthe second file DEP 243, includes a depth map stream.

In the example shown in FIG. 2, the fourth AV stream file (01000.ssif)244A and the fifth AV stream file (02000.ssif) 244B are both a file SS.In this way, files SS are located directly below the SSIF directory 244.The fourth AV stream file, i.e. the file SS 244A, shares a main TS, andin particular a base-view video stream, with the file 2D 241 and sharesa sub-TS, in particular a right-view video stream, with the first fileDEP 242. The fifth AV stream file, i.e. the second file SS 244B, sharesa main TS, and in particular a base-view video stream, with the firstfile 2D 241 and shares a sub-TS, in particular a depth map stream, withthe third file DEP 243.

Three types of clip information files, (01000.clpi) 231, (02000.clpi)232, and (03000.clpi) 233 are located in the CLIPINF directory 230. A“clip information file” is a file associated on a one-to-one basis witha file 2D and a file DEP and in particular contains an entry map foreach file. An “entry map” is a correspondence table between thepresentation time for each scene represented by the file and the addresswithin each file at which the scene is recorded. Among the clipinformation files, a clip information file associated with a file 2D isreferred to as a “2D clip information file”, and a clip information fileassociated with a file DEP is referred to as a “dependent-view clipinformation file”. In the example shown in FIG. 2, the first clipinformation file (01000.clpi) 231 is a 2D clip information file and isassociated with the file 2D 241. The second clip information file(02000.clpi) 232 and the third clip information file (03000.clpi) 233are both a dependent-view clip information file, and are associated withthe first file DEP 242 and the second file DEP 243, respectively.

Three types of playlist files, (00001.mpls) 221, (00002.mpls) 222, and(00003.mpls) 223 are located in the PLAYLIST directory 220. A “playlistfile” is a file that specifies the playback path of an AV stream file,i.e. the part of an AV stream file for playback, and the order ofplayback. The types of playlist files are a 2D playlist file and a 3Dplaylist file. A “2D playlist file” specifies the playback path of afile 2D. A “3D playlist file” specifies, for a playback device in 2Dplayback mode, the playback path of a file 2D, and for a playback devicein 3D playback mode, the playback path of a file SS. As shown in theexample in FIG. 2, the first playlist file (00001.mpls) 221 is a 2Dplaylist file and specifies the playback path of the file 2D 241. Thesecond playlist file (00002.mpls) 222 is a 3D playlist file thatspecifies, for a playback device in 2D playback mode, the playback pathof the file 2D 241, and for a 3D playback device in L/R mode, theplayback path of the file SS 244A. The third playlist file (00003.mpls)223 is a 3D playlist file that specifies, for a playback device in 2Dplayback mode, the playback path of the file 2D 241, and for a 3Dplayback device in depth mode, the playback path of the second file SS244B.

A BD-J object file (XXXXX.bdjo) 251 is located in the BDJO directory250. The BD-J object file 251 includes a single BD-J object. The BD-Jobject is a bytecode program to cause a Java virtual machine mounted onthe playback device 102 to play back a title and render graphics images.The BD-J object is written in a compiler language such as Java or thelike. The BD-J object includes an application management table andidentification information for the playlist file to which is referred.The “application management table” is a list of the Java applicationprograms to be executed by the Java virtual machine and their period ofexecution, i.e. lifecycle. The “identification information of theplaylist file to which is referred” identifies a playlist file thatcorresponds to a title to be played back. The Java virtual machine callsa BD-J object in response to a user operation or an application programand executes the Java application program according to the applicationmanagement table included in the BD-J object. Consequently, the playbackdevice 102 dynamically changes the progress of the video for each titleplayed back, or causes the display device 103 to display graphics imagesindependently of the title video.

A JAR file (YYYYY.jar) 261 is located in the JAR directory 260. The JARdirectory 261 generally includes a plurality of actual Java applicationprograms to be executed in accordance with the application managementtable shown in the BD-J object. A “Java application program” is abytecode program written in a compiler language such as Java or thelike, as is the BD-J object. Types of Java application programs includeprograms causing the Java virtual machine to perform playback of a titleand programs causing the Java virtual machine to render graphics images.The JAR file 261 is a Java archive file, and when it is read by theplayback device 102, it is loaded in internal memory. In this way, aJava application program is stored in memory.

<<Structure of Multiplexed Stream Data>>

FIG. 3A is a list of elementary streams multiplexed in a main TS on aBD-ROM disc 101. The main TS is a digital stream in MPEG-2 TransportStream (TS) format and is included in the file 2D 241 shown in FIG. 2.As shown in FIG. 3A, the main TS includes a primary video stream 301,primary audio streams 302A and 302B, and presentation graphics (PG)streams 303A and 303B. The main TS may additionally include aninteractive graphics (IG) stream 304, a secondary audio stream 305, anda secondary video stream 306.

The primary video stream 301 represents the primary video of a movie,and the secondary video stream 306 represents secondary video of themovie. The primary video is the main video pertaining to the content,such as the main feature of a movie, and is displayed on the entirescreen, for example. On the other hand, the secondary video is displayedon the screen simultaneously with the primary video with the use, forexample, of a picture-in-picture method, so that the secondary videoimages are displayed in a smaller window within the primary videoimages. The primary video stream 301 and the secondary video stream 306are both a base-view video stream. Each of the video streams 301 and 306is encoded by a video compression encoding method, such as MPEG-2,MPEG-4 AVC, or SMPTE VC-1.

The primary audio streams 302A and 302B represent the primary audio ofthe movie. In this case, the two primary audio streams 302A and 302B arein different languages. The secondary audio stream 305 representssecondary audio to be mixed with the primary audio, such as soundeffects accompanying operation of an interactive screen. Each of theaudio streams 302A, 302B, and 305 is encoded by a method such as AC-3,Dolby Digital Plus (“Dolby Digital” is a registered trademark), MeridianLossless Packing™ (MLP), Digital Theater System™ (DTS), DTS-HD, orlinear Pulse Code Modulation (PCM).

Each of the PG streams 303A and 303B represents graphics images, such assubtitles formed by graphics, to be displayed superimposed on the videoimages represented by the primary video stream 301. The two PG streams303A and 303B represent, for example, subtitles in a different language.The IG stream 304 represents Graphical User Interface (GUI) graphicelements, and the arrangement thereof, for constructing an interactivescreen on the screen 131 in the display device 103.

The elementary streams 301-306 are identified by packet identifiers(PIDs). PIDs are assigned, for example, as follows. Since one main TSincludes only one primary video stream, the primary video stream 301 isassigned a hexadecimal value of 0x1011. When up to 32 other elementarystreams can be multiplexed by type in one main TS, the primary audiostreams 302A and 302B are each assigned any value from 0x1100 to 0x111F.The PG streams 303A and 303B are each assigned any value from 0x1200 to0x121F. The IG stream 304 is assigned any value from 0x1400 to 0x141F.The secondary audio stream 305 is assigned any value from 0x1A00 to0x1A1F. The secondary video stream 306 is assigned any value from 0x1B00to 0x1B1F.

FIG. 3B is a list of elementary streams multiplexed in a sub-TS on aBD-ROM disc 101. The sub-TS is multiplexed stream data in MPEG-2 TSformat and is included in the file DEP 242 shown in FIG. 2. As shown inFIG. 3B, the sub-TS includes two primary video streams 311R and 311D.311R is a right-view video stream, whereas 311D is a depth map stream.Note that the primary video streams 311R and 311D may be multiplexedinto files DEP 242 and 243, which are different files, separately. Whenthe primary video stream 301 in the main TS represents the left view of3D video images, the right-view video stream 311R represents the rightview of the 3D video images. The depth map stream 311D represents 3Dvideo images in combination with the primary video stream 301 in themain TS. Additionally, the sub TS may include secondary video streams312R and 312D. 312R is a right-view video stream, whereas 312D is adepth map stream. When the secondary video stream 306 in the main TSrepresents the left view of 3D video images, the right-view video stream312R represents the right view of the 3D video images. The depth mapstream 312D represents 3D video images in combination with the secondaryvideo stream 306 in the main TS.

PIDs are assigned to the elementary streams 311R, . . . , 312D asfollows, for example. The primary video streams 311R and 311D arerespectively assigned values of 0x1012 and 0x1013. When up to 32 otherelementary streams can be multiplexed by type in one sub-TS, thesecondary video streams 312R and 312D are assigned any value from 0x1B20to 0x1B3F.

FIG. 4 is a schematic diagram showing the arrangement of TS packets inthe multiplexed stream data 400. The main TS and sub TS share thispacket structure. In the multiplexed stream data 400, the elementarystreams 401, 402, 403, and 404 are respectively converted into sequencesof TS packets 421, 422, 423, and 424. For example, in the video stream401, each frame 401A or each field is first converted into onePacketized Elementary Stream (PES) packet 411. Next, each PES packet 411is generally converted into a plurality of TS packets 421. Similarly,the audio stream 402, PG stream 403, and IG stream 404 are respectivelyfirst converted into a sequence of PES packets 412, 413, and 414, afterwhich they are converted into a sequence of TS packets 422, 423, and424. Finally, the TS packets 421, 422, 423, and 424 obtained from theelementary streams 401, 402, 403, and 404 are time-multiplexed into onepiece of stream data, i.e. the main TS 400.

FIG. 5B is a schematic diagram showing a TS packet sequence constitutingmultiplexed stream data. Each TS packet 501 is 188 bytes long. As shownin FIG. 5B, each TS packet 501 includes a TS header 501H and either, orboth, a TS payload 501P and an adaptation field (hereinafter abbreviatedas “AD field”) 501A. The TS payload 501P and AD field 501A togetherconstitute a 184 byte long data area. The TS payload 501P is used as astorage area for a PES packet. The PES packets 411-414 shown in FIG. 4are typically divided into a plurality of parts, and each part is storedin a different TS payload 501P. The AD field 501A is an area for storingstuffing bytes (i.e. dummy data) when the amount of data in the TSpayload 501P does not reach 184 bytes. Additionally, when the TS packet501 is, for example, a PCR as described below, the AD field 501A is usedto store such information. The TS header 501H is a four-byte long dataarea.

FIG. 5A is a schematic diagram showing the data structure of a TS header501H. As shown in FIG. 5A, the TS header 501H includes TS priority(transport_priority) 511, PID 512, and AD field control (adaptationfield control) 513. The PID 512 indicates the PID for the elementarystream whose data is stored in the TS payload 501P of the TS packet 501containing the PID 512. The TS priority 511 indicates the degree ofpriority of the TS packet 501 among the TS packets that share the valueindicated by the PID 512. The AD field control 513 indicates whether theTS packet 501 contains an AD field 501A and/or a TS payload 501P. Forexample, if the AD field control 513 indicates “1”, then the TS packet501 does not include an AD field 501A but includes a TS payload 501P. Ifthe AD field control 513 indicates “2”, then the reverse is true. If theAD field control 513 indicates “3”, then the TS packet 501 includes bothan AD field 501A and a TS payload 501P.

FIG. 5C is a schematic diagram showing the formation of a source packetsequence composed of the TS packet sequence for multiplexed stream data.As shown in FIG. 5C, each source packet 502 is 192 bytes long andincludes one TS packet 501, shown in FIG. 5B, and a four-byte longheader (TP_Extra_Header) 502H. When the TS packet 501 is recorded on theBD-ROM disc 101, a source packet 502 is constituted by attaching aheader 502H to the TS packet 501. The header 502H includes an ATS(Arrival_Time_Stamp). The “ATS” is time information used by the playbackdevice 102 as follows. When a source packet 502 is sent from the BD-ROMdisc 101 to a system target decoder in the playback device 102, the TSpacket 502P is extracted from the source packet 502 and transferred to aPID filter in the system target decoder. The ATS in the header 502Hindicates the time at which this transfer is to begin. The “systemtarget decoder” is a device that decodes multiplexed stream data oneelementary stream at a time. Details regarding the system target decoderand its use of the ATS are provided below.

FIG. 5D is a schematic diagram of a sector group, in which a sequence ofsource packets 502 is consecutively recorded, in the volume area 202B ofthe BD-ROM disc 101. As shown in FIG. 5D, 32 source packets 502 arerecorded at a time as a sequence in three consecutive sectors 521, 522,and 523. This is because the data amount for 32 source packets, i.e. 192bytes×32=6144 bytes, is the same as the total size of three sectors,i.e. 2048 bytes×3=6144 bytes. 32 source packets 502 that are recorded inthis way in three consecutive sectors 521, 522, and 523 are referred toas an “aligned unit” 520. The playback device 102 reads source packets502 from the BD-ROM disc 101 by each aligned unit 520, i.e. 32 sourcepackets at a time. Also, the sector group 521, 522, 523, . . . isdivided into 32 pieces in order from the top, and each forms one errorcorrection code block 530. The BD-ROM drive 121 performs errorcorrection processing for each ECC block 530.

<<Data Structure of PG Stream>>

FIG. 6 is a schematic diagram showing the data structure of the PGstream 600. As shown in FIG. 6, the PG stream 600 includes a pluralityof data entries #1, #2, . . . . Each data entry represents a displayunit (display set) of the PG stream 600, and is composed of data that isnecessary for the playback device 102 to form one graphics plane. Here“graphics plane” is plane data generated from graphics data representing2D graphics images. “Plane data” is a two-dimensional array of pixeldata. The size of the array is the same as the resolution of the videoframe. A set of pixel data is formed by a combination of a chromaticcoordinate value and an α value (opaqueness). The chromatic coordinatevalue is expressed as an RGB value or a YCrCb value. Types of graphicsplanes include a PG plane, IG plane, image plane, and On-Screen Display(OSD) plane. A PG plane is generated from a PG stream in the main TS. AnIG plane is generated from an IG stream in the main TS. An image planeis generated in accordance with a BD-J object. An OSD plane is generatedin accordance with firmware in the playback device 102.

Referring again to FIG. 6, each data entry includes a plurality offunctional segments. These functional segments include, in order fromstart, a Presentation Control Segment (PCS), Window Define Segment(WDS), Pallet Define Segment (PDS), and Object Define Segment (ODS).

WDS defines a rectangular region inside the graphics plane, i.e. awindow. More specifically, WDS includes a window ID 611, a windowposition 612, and a window size 613. The window ID 611 is identificationinformation (ID) of the WDS. The window position 612 indicates theposition of a window in the graphics plane by, for example, coordinatesof the upper-left corner of the window. The window size 613 indicatesthe height and width of the window.

PDS defines the correspondence between a predetermined type of color IDand a chromatic coordinate value (for example, luminance Y,red-difference Cr, blue-difference Cb, opaqueness a). Specifically, thePDS includes a pallet ID 621 and a color look-up table (CLUT) 622. Thepallet ID 621 is the ID of the PDS. The CLUT 622 is a table showing alist of colors that can be used in rendering the graphics object. In theCLUT 622, 256 colors can be registered, wherein color IDs from “0” to“255” are assigned to the respective 256 colors. Note that color ID=255is constantly assigned to “colorless transparent”.

In general, a plurality of ODSs represent one graphics object. “Graphicsobject” is data that represents a graphics image by the correspondencebetween a pixel code and a color ID. The graphics object is divided intoparts after it is compressed by the run-length coding method, and theparts are distributed to each ODS. Each ODS further includes an objectID, namely an ID of the graphics object.

The PCS shows details of a display set that belongs to the same dataentry, and in particular defines a screen structure that uses graphicsobjects. Types of screen structure include Cut-In/Out, Fade-In/Out,Color Change, Scroll, and Wipe-In/Out. Specifically, the PCS includes anobject display position 601, cropping information 602, reference windowID 603, reference pallet ID 604, and reference object ID 605. The objectdisplay position 601 indicates a position in the graphics plane at whichthe graphics object is to be displayed, by, for example, coordinates ofthe upper-left corner of an area in which the graphics object is to bedisplayed. The cropping information 602 indicates the range of arectangular part that is to be cut out of the graphics object by thecropping process. The range is defined by, for example, coordinates ofthe upper-left corner, height and width. Actually, the part can berendered at a position indicated by the object display position 601. Thereference window ID 603, reference pallet ID 604, and reference objectID 605 indicate IDs of the WDS, PDS, and graphics object that are to bereferred to in the graphics object rendering process, respectively. Thecontent provider indicates the structure of the screen to the playbackdevice 102 by using these parameters in the PCS. This allows theplayback device 102 to realize a display effect whereby “a certainsubtitle gradually disappears, and the next subtitle is displayed”.

<<Data Structure of IG Stream>>

Referring yet again to FIG. 4, the IG stream 404 includes an InteractiveComposition Segment (ICS), PDS, and ODS. PDS and ODS are the samefunctional segments as included in the PG stream 403. In particular, agraphics object that includes an ODS represents a GUI graphic element,such as a button, pop-up menu, etc., that forms an interactive screen.An ICS defines interactive operations that use these graphics objects.Specifically, an ICS defines the states that each graphics object, suchas a button, pop-up menu, etc. can take when changed in response to useroperation, states such as normal, selected, and active. An ICS alsoincludes button information. Button information includes a command thatthe playback device is to perform when the user performs a certainoperation on the button or the like.

<<Data Structure of Video Stream>>

Each of the pictures included in the video stream represents one frameor one field and is compressed by a video compression encoding method,such as MPEG-2, MPEG-4 AVC, etc. This compression uses the picture'sspatial or temporal redundancy. Here, picture encoding that only usesthe picture's spatial redundancy is referred to as “intra-pictureencoding”. On the other hand, picture encoding that uses temporalredundancy, i.e. the similarity between data for a plurality of picturesdisplayed sequentially, is referred to as “inter-picture predictiveencoding”. In inter-picture predictive encoding, first, a pictureearlier or later in presentation time is assigned to the picture to beencoded as a reference picture. Next, a motion vector is detectedbetween the picture to be encoded and the reference picture, and thenmotion compensation is performed on the reference picture using themotion vector. Furthermore, the difference value between the pictureobtained by motion compensation and the picture to be encoded is sought,and spatial redundancy is removed using the difference value. In thisway, the amount of data for each picture is compressed.

FIG. 7 is a schematic diagram showing the pictures for a base-view videostream 701 and a right-view video stream 702 in order of presentationtime. As shown in FIG. 7, the base-view video stream 701 includespictures 710, 711, 712, . . . , 719 (hereinafter “base-view pictures”),and the right-view video stream 702 includes pictures 720, 721, 722, . .. , 729 (hereinafter “right-view pictures”). The base-view pictures710-719 are typically divided into a plurality of GOPs 731 and 732. A“GOP” refers to a sequence of pictures having an I picture at the top ofthe sequence. In addition to an I picture, a GOP typically includes Ppictures and B pictures. Here “I (Intra) picture” refers to a picturecompressed by the intra-picture encoding. “P (Predictive) picture”refers to a picture compressed by the inter-picture predictive encodingby using another picture whose presentation time is before thepresentation time of the picture as a reference picture. “B(Bidirectionally Predictive) picture” refers to a picture compressed bythe inter-picture predictive encoding by using two pictures whosepresentation times are before or after the presentation time of thepicture as reference pictures. In particular, a B picture which is usedas a reference picture for another picture in the inter-picturepredictive encoding is referred to as “Br (reference B) picture”.

In the example shown in FIG. 7, the base-view pictures in the GOPs 731and 732 are compressed in the following order. In the first GOP 731, thetop base-view picture is compressed as I₀ picture 710. The subscriptednumber indicates the serial number allotted to each picture in the orderof presentation time. Next, the fourth base-view picture is compressedas P₃ picture 713 using I₀ picture 710 as a reference picture. Thearrows shown in FIG. 7 indicate that the picture at the head of thearrow is a reference picture for the picture at the tail of the arrow.Next, the second and third base-view pictures are respectivelycompressed as Br₁ picture 711 and Br₂ picture 712, using both I₀ picture710 and P₃ picture 713 as reference pictures. Furthermore, the seventhbase-view picture is compressed as P₆ picture 716 using P₃ picture 713as a reference picture. Next, the fourth and fifth base-view picturesare respectively compressed as Br₄ picture 714 and Br₅ picture 715,using both P₃ picture 713 and P₆ picture 716 as reference pictures.Similarly, in the second GOP 732, the top base-view picture is firstcompressed as I₇ picture 717. Next, the third base-view picture iscompressed as P₉ picture 719 using I₇ picture 717 as a referencepicture. Subsequently, the second base-view picture is compressed as Br₈picture 718 using both I₇ picture 717 and P₉ picture 719 as referencepictures.

In the base-view video stream 701, each GOP 731 and 732 always containsan I picture at the top, and thus base-view pictures can be decoded GOPby GOP. For example, in the first GOP 731, the I₀ picture 710 is firstdecoded independently. Next, the P₃ picture 713 is decoded using thedecoded I₀ picture 710. Then the Br₁ picture 711 and Br₂ picture 712 aredecoded using both the decoded I₀ picture 710 and P₃ picture 713. Thesubsequent picture group 714, 715, . . . is similarly decoded. In thisway, the base-view video stream 701 can be decoded independently andfurthermore can be randomly accessed in units of GOPs.

As further shown in FIG. 7, the right-view pictures 720-729 arecompressed by inter-picture predictive encoding. However, the encodingmethod differs from the encoding method for the base-view pictures710-719, since in addition to redundancy in the temporal redundancy ofvideo images, redundancy between the left and right-video images is alsoused. Specifically, as shown by the arrows in FIG. 7, the referencepicture for each of the right-view pictures 720-729 is not selected fromthe right-view video stream 702, but rather from the base-view videostream 701. In particular, the presentation time is substantially thesame for each of the right-view pictures 720-729 and the correspondingbase-view picture selected as a reference picture. These picturesrepresent a right view and a left view for the same scene of a 3D videoimage, i.e. a parallax video image. The right-view pictures 720-729 andthe base-view pictures 710-719 are thus in one-to-one correspondence. Inparticular, the GOP structure is the same between these pictures.

In the example shown in FIG. 7, the top right-view picture in the firstGOP 731 is compressed as P₀ picture 720 using I₀ picture 710 in thebase-view video stream 701 as a reference picture. These pictures 710and 720 represent the left view and right view of the top frame in the3D video images. Next, the fourth right-view picture is compressed as P₃picture 723 using P₃ picture 713 in the base-view video stream 701 andP₀ picture 720 as reference pictures. Next, the second right-viewpicture is compressed as B₁ picture 721, using Br₁ picture 711 in thebase-view video stream 701 in addition to P₀ picture 720 and P₃ picture723 as reference pictures. Similarly, the third right-view picture iscompressed as B₂ picture 722, using Br₂ picture 712 in the base-viewvideo stream 701 in addition to P₀ picture 720 and P₃ picture 730 asreference pictures. For each of the remaining right-view pictures724-729, a base-view picture with a presentation time substantially thesame as the right-view picture is similarly used as a reference picture.

The revised standards for MPEG-4 AVC/H.264, called Multiview VideoCoding (MVC), are known as a video compression encoding method thatmakes use of correlation between left and right-video images asdescribed above. MVC was created in July of 2008 by the Joint Video Team(JVT), a joint project between ISO/IEC MPEG and ITU-T VCEG, and is astandard for collectively encoding video that can be seen from aplurality of perspectives. With MVC, not only is temporal similarity invideo images used for inter-video predictive encoding, but so issimilarity between video images from differing perspectives. This typeof predictive encoding has a higher video compression ratio thanpredictive encoding that individually compresses data of video imagesseen from each perspective.

As described above, a base-view picture is used as a reference picturefor compression of each of the right-view pictures 720-729. Therefore,unlike the base-view video stream 701, the right-view video stream 702cannot be decoded independently. On the other hand, however, thedifference between parallax video images is generally very small; thatis, the correlation between the left view and the right view is high.Accordingly, the right-view pictures generally have a significantlyhigher compression rate than the base-view pictures, meaning that theamount of data is significantly smaller.

The depth maps included in a depth map stream are in one-to-onecorrespondence with the base-view pictures 710-719 and each represent adepth map for the 2D video image in the corresponding base-view picture.The depth maps are compressed by a video compression encoding method,such as MPEG-2, MPEG-4 AVC, etc., in the same way as the base-viewpictures 710-719. In particular, inter-picture predictive encoding isused in this encoding method. In other words, each depth map iscompressed using another depth map as a reference picture. Furthermore,the depth map stream is divided into units of GOPs in the same way asthe base-view video stream 701, and each GOP always contains an Ipicture at the top. Accordingly, depth maps can be decoded GOP by GOP.However, since a depth map itself is only information representing thedepth of each part of a 2D video image pixel by pixel, the depth mapstream cannot be used independently for playback of video images.

For example, as in the two primary video streams 311R and 311D shown inFIG. 3B, the right-view video stream and depth map stream thatcorrespond to the same base-view video stream are compressed with thesame encoding method. For example, if the right-view video stream isencoded in MVC format, the depth map stream is also encoded in MVCformat. In this case, during playback of 3D video images, the playbackdevice 102 can smoothly switch between L/R mode and depth mode, whilemaintaining a constant encoding method.

FIG. 8 is a schematic diagram showing details on a data structure of avideo stream 800. This data structure is substantially the same for thebase-view video stream and the dependent-view video stream. As shown inFIG. 8, the video stream 800 is generally composed of a plurality ofvideo sequences #1, #2, . . . A “video sequence” is a combination ofpictures 811, 812, 813, 814, . . . that constitute a single GOP 810 andto which additional information, such as a header, has been individuallyattached. The combination of this additional information and a pictureis referred to as a “video access unit (VAU)”. That is, in the GOPs 810and 820, a single VAU #1, #2, . . . is formed for each picture. Eachpicture can be read from the video stream 800 in units of VAUs.

FIG. 8 further shows the structure of VAU #1 831 located at the top ofeach video sequence in the base-view video stream. The VAU #1 831includes an access unit (AU) identification code 831A, sequence header831B, picture header 831C, supplementary data 831D, and compressedpicture data 831E. Except for not including a sequence header 831B, VAUsfrom the second VAU #2 on have the same structure as VAU #1 831. The AUidentification code 831A is a predetermined code indicating the top ofthe VAU #1 831. The sequence header 831B, also referred to as a GOPheader, includes an identification number for the video sequence #1which includes the VAU #1 831. The sequence header 831B further includesinformation shared by the whole GOP 810, e.g. resolution, frame rate,aspect ratio, and bit rate. The picture header 831C indicates its ownidentification number, the identification number for the video sequence#1, and information necessary for decoding the picture, such as the typeof encoding method. The supplementary data 831D includes additionalinformation regarding matters other than the decoding of the picture,for example closed caption text information, information on the GOPstructure, and time code information. In particular, the supplementarydata 831D includes decoding switch information (details provided below).A plurality of pieces of supplementary data 831D may be set in one VAUdepending on the type of data contained therein. The compressed picturedata 831E includes a base-view picture. Additionally, the VAU #1 831 mayinclude any or all of padding data 831F, a sequence end code 831G, and astream end code 831H as necessary. The padding data 831F is dummy data.By adjusting the size of the padding data 831F in conjunction with thesize of the compressed picture data 831E, the bit rate of the VAU #1 831can be maintained at a predetermined value. The sequence end code 831Gindicates that the VAU #1 831 is located at the end of the videosequence #1. The stream end code 831H indicates the end of the base-viewvideo stream 800.

FIG. 8 also shows the structure of a VAU #1 832 located at the top ofeach video sequence in the dependent-view video stream. The VAU #1 832includes a sub-AU identification code 832A, sub-sequence header 832B,picture header 832C, supplementary data 832D, and compressed picturedata 832E. Except for not including a sub-sequence header 832B, VAUsfrom the second VAU #2 on have the same structure as VAU #1 832. Thesub-AU identification code 832A is a predetermined code indicating thetop of the VAU #1 832. The sub-sequence header 832B includes anidentification number for the video sequence #1 which includes the VAU#1 832. The sub-sequence header 832B further includes information sharedby the whole GOP 810, e.g. resolution, frame rate, aspect ratio, and bitrate. These values are the same as the values set for the correspondingGOP in the base-view video stream, i.e. the values shown by the sequenceheader 831B in the VAU #1 831. The picture header 832C indicates its ownidentification number, the identification number for the video sequence#1, and information necessary for decoding the picture, such as the typeof encoding method. The supplementary data 832D includes only offsetmetadata (details provided below). Here the supplementary data 832D isone type of supplementary data, and there is another type ofsupplementary data that includes additional information regardingmatters other than the decoding of the picture, for example, closedcaption text information, information on the GOP structure, time codeinformation, and decoding switch information. The compressed picturedata 832E includes a dependent-view picture. Additionally, the VAU #1832 may include any or all of padding data 832F, a sequence end code832G, and a stream end code 832H as necessary. The padding data 832F isdummy data. By adjusting the size of the padding data 832F inconjunction with the size of the compressed picture data 832E, the bitrate of the VAU #1 832 can be maintained at a predetermined value. Thesequence end code 832G indicates that the VAU #1 832 is located at theend of the video sequence #1. The stream end code 832H indicates the endof the dependent-view video stream 800.

The specific content of each component in a VAU differs according to theencoding method of the video stream 800. For example, when the encodingmethod is MPEG-4 AVC or MVC, the components in the VAUs shown in FIG. 8are composed of a single Network Abstraction Layer (NAL) unit.Specifically, the AU identification code 831A, sequence header 831B,picture header 831C, supplementary data 831D, compressed picture data831E, padding data 831F, sequence end code 831G, and stream end code831H respectively correspond to an Access Unit (AU) delimiter, SequenceParameter Set (SPS), Picture Parameter Set (PPS), SupplementalEnhancement Information (SEI), View Component, Filler Data, End ofSequence, and End of Stream. In particular, in the VAU #1 832, thesupplementary data 832D including the offset metadata is composed of oneNAL unit, wherein the NAL unit does not include data other than theoffset metadata.

FIG. 9 is a schematic diagram showing details on a method for storing avideo stream 901 into a PES packet sequence 902. This storage method isthe same for the base-view video stream and the dependent-view videostream. As shown in FIG. 9, in the actual video stream 901, pictures aremultiplexed in the order of encoding, not in the order of presentationtime. For example, in the VAUs in the base-view video stream, as shownin FIG. 9, I₀ picture 910, P₃ picture 911, B₁ picture 912, B₂ picture913, . . . are stored in order from the top. The subscripted numberindicates the serial number allotted to each picture in order ofpresentation time. I₀ picture 910 is used as a reference picture forencoding P₃ picture 911, and both I₀ picture 910 and P₃ picture 911 areused as reference pictures for encoding B₁ picture 912 and B₂ picture913. Each of these VAUs is stored as a different PES packet 920, 921,922, 923, . . . . Each PES packet 920, . . . includes a PES payload 920Pand a PES header 920H. Each VAU is stored in a PES payload 920P. EachPES header 920H includes a presentation time, (Presentation Time-Stamp,or PTS), and a decoding time (Decoding Time-Stamp, or DTS), for thepicture stored in the PES payload 920P in the same PES packet 920.

As with the video stream 901 shown in FIG. 9, the other elementarystreams shown in FIGS. 3 and 4 are stored in PES payloads in a sequenceof PES packets. Furthermore, the PES header in each PES packet includesthe PTS for the data stored in the PES payload for the PES packet.

FIG. 10 is a schematic diagram showing correspondence between PTSs andDTSs assigned to each picture in a base-view video stream 1001 and adependent-view video stream 1002. As shown in FIG. 10, between the videostreams 1001 and 1002, the same PTSs and DTSs are assigned to a pair ofpictures representing the same frame or field in a 3D video image. Forexample, the top frame or field in the 3D video image is rendered from acombination of I₁ picture 1011 in the base-view video stream 1001 and P₁picture 1021 in the dependent-view video stream 1002. Accordingly, thePTS and DTS for these two pictures 1011 and 1021 are the same. Thesubscripted numbers indicate the serial number allotted to each picturein the order of DTSs. Also, when the dependent-view video stream 1002 isa depth map stream, P₁ picture 1021 is replaced by an I picturerepresenting a depth map for the I₁ picture 1011. Similarly, the PTS andDTS for the pair of second pictures in the video streams 1001 and 1002,i.e. P₂ pictures 1012 and 1022, are the same. The PTS and DTS are boththe same for the pair of third pictures in the video streams 1001 and1002, i.e. Br_(a) picture 1013 and B₃ picture 1023. The same is alsotrue for the pair Br₄ picture 1014 and B₄ picture 1024.

A pair of VAUs that include pictures for which the PTS and DTS are thesame between the base-view video stream 1001 and the dependent-viewvideo stream 1002 is called a “3D VAU”. Using the allocation of PTSs andDTSs shown in FIG. 10, it is easy to cause the decoder in the playbackdevice 102 in 3D playback mode to process the base-view video stream1001 and the dependent-view video stream 1002 in parallel in units of 3DVAUs. In this way, the decoder definitely processes a pair of picturesrepresenting the same frame or field in a 3D video image in parallel.Furthermore, the sequence header in the 3D VAU at the top of each GOPincludes the same resolution, the same frame rate, and the same aspectratio. In particular, this frame rate is equal to the value when thebase-view video stream 1001 is decoded independently in 2D playbackmode.

[Offset Metadata]

FIG. 11 is a schematic diagram showing a data structure of offsetmetadata 1110 included in a dependent-view video stream 1100. FIG. 12 isa table showing syntax of this offset metadata 1110. As shown in FIG.11, the offset metadata 1110 is stored in the supplementary data 1101 ofVAU #1 located at the top of each video sequence (i.e., each GOP). Asshown in FIGS. 11 and 12, the offset metadata 1110 includes a PTS 1111,offset sequence IDs 1112, and offset sequences 1113. The PTS 1111 is thesame as a PTS of a frame represented by compressed picture data in VAU#1, namely a PTS of the first frame of each GOP.

The offset sequence IDs 1112 are serial numbers 0, 1, 2, . . . , Mallotted in order to the offset sequences 1313. The letter M representsan integer greater than or equal to 1 and indicates the total number ofoffset sequences 1113 (number of offset sequence). An offset sequence ID1112 is allocated to each graphics plane to be combined in a videoplane. In this way, an offset sequence 1113 is associated with eachgraphics plane. Here a “video plane” refers to plane data generated froma picture included in a video sequence, namely to a two-dimensionalarray of pixel data. The size of the array is the same as the resolutionof the video frame. A set of pixel data is formed by a combination of achromatic coordinate value (an RGB value or a YCrCb value) and an αvalue.

Each offset sequence 1113 is a correspondence table between framenumbers 1121 and offset information 1122 and 1123. Frame numbers 1121are serial numbers 1, 2, . . . , N allocated in order of presentation toframes #1, #2, . . . , N represented by a single video sequence (forexample, video sequence #1). In FIG. 11, the frame number 1121 isrepresented as an integer variable “i”. The letter N represents aninteger greater than or equal to one and indicates the total number offrames included in the video sequence (number of displayed frames inGOP). The pieces of offset information 1122 and 1123 are controlinformation defining offset control for a single graphics plane.

“Offset control” refers to a process to provide left and right offsetsfor the horizontal coordinates in a graphics plane and combine theresulting planes respectively with the base-view video plane anddependent-view video plane. “Providing horizontal offsets to a graphicsplane” refers to horizontally shifting each piece of pixel data in thegraphics plane. From a single graphics plane, this generates a pair ofgraphics planes representing a left view and a right view. Thepresentation position of each element in the 2D graphics images playedback from this pair of planes is shifted to the left or right from theoriginal presentation position. The viewer is made to perceive a pair ofa left view and a right view as a single 3D graphics image due to thebinocular parallax produced by these shifts.

An offset is determined by a direction and a size. Accordingly, as shownin FIGS. 11 and 12, each piece of offset information includes an offsetdirection (Plane_offset_direction) 1122 and an offset value(Plane_offset_value) 1123. The offset direction 1122 indicates whether a3D graphics image is closer to the viewer than the screen or furtherback. Whether the presentation position in the left view and the rightview is shifted to the left or to the right from the originalpresentation position of the 2D graphics image depends on the value ofthe offset direction 1122. The offset value 1123 indicates the number ofhorizontal pixels of the distance between the original presentationposition of the 2D graphics image and the presentation position of eachof the left view and the right view.

FIGS. 13A and 13B are schematic diagrams showing offset controls for aPG plane 1310 and IG plane 1320 respectively. Via these offset controls,two types of graphics planes, 1310 and 1320, are respectively combinedwith the left-view video plane 1301 and the right-view video plane 1302.A “left-view/right-view video plane” refers to a video plane thatrepresents a left view/right view and is generated from a combination ofthe base-view video stream and the dependent-view video stream. In thefollowing description, it is assumed that a subtitle 1311 indicated bythe PG plane 1310 is displayed closer than the screen, and a button 1321indicated by the IG plane 1320 is displayed further back than thescreen.

As shown in FIG. 13A, a right offset is provided to the PG plane 1310.Specifically, the position of each piece of pixel data in the PG plane1310 is first shifted to the right (virtually) from the correspondingposition of the pixel data in the left-view video plane 1301 by a numberof pixels SFP equal to the offset value. Next, a strip 1512 (virtually)protruding from the right edge of the range of the left-view video plane1301 is “cut off” from the right edge of the PG plane 1310. In otherwords, the pixel data for this region 1312 is discarded. Conversely, atransparent strip 1513 is added to the left edge of the PG plane 1310.The width of this strip 1513 is the width of the strip 1512 at the rightedge; i.e. the width is the same as the offset value SFP. A PG planerepresenting the left view is thus generated from the PG plane 1310 andcombined with the left-view video plane 1301. In particular, in thisleft-view PG plane, the presentation position of the subtitle 1311 isshifted to the right from the original presentation position by theoffset value SFP.

Conversely, a left offset is provided to the IG plane 1320.Specifically, the position of each piece of pixel data in the IG plane1320 is first shifted to the left (virtually) from the correspondingposition of the pixel data in the left-view video plane 1301 by a numberof pixels SFI equal to the offset value. Next, a strip 1322 (virtually)protruding from the left edge of the range of the left-view video plane1310 is cut off from the left edge of the IG plane 1320. Conversely, atransparent strip 1323 is added to the right edge of the IG plane 1320.The width of this strip 1323 is the width of the strip 1322 at the leftedge; i.e. the width is the same as the offset value SFI. An IG planerepresenting the left view is thus generated from the IG plane 1320 andcombined with the left-view video plane 1301. In particular, in thisleft-view IG plane, the presentation position of the button 1321 isshifted to the left from the original presentation position by theoffset value SFI.

As shown in FIG. 13B, a left offset is provided to the PG plane 1310,and a right offset is added to the IG plane 1320. In other words, theabove operations are performed in reverse for the PG plane 1310 and theIG plane 1320. As a result, plane data representing the right view isgenerated from the plane data 1310 and 1320 and combined with theright-view video plane 1320. In particular, in the right-view PG plane,the presentation position of the subtitle 1311 is shifted to the leftfrom the original presentation position by the offset value SFP. On theother hand, in the right-view IG plane, the presentation position of thebutton 1321 is shifted to the right from the original presentationposition by the offset value SFI.

FIG. 13C is a schematic diagram showing 3D graphics images that a viewer1330 is made to perceive from 2D graphics images represented by graphicsplanes shown in FIGS. 13A and 13B. When the 2D graphics imagesrepresented by these graphics planes are alternately displayed on thescreen 1340, the viewer 1330 perceives the subtitle 1331 to be closerthan the screen 1340 and the button 1332 to be further back than thescreen 1340, as shown in FIG. 13C. The distance between the 3D graphicsimages 1331 and 1332 and the screen 1340 can be adjusted via the offsetvalues SFP and SFI.

FIGS. 14A and 14B are graphs showing examples of offset sequences. Inthese graphs, the offset value is positive when the offset direction istoward the viewer from the screen. FIG. 14A is an enlargement of thegraph for the presentation period of the first GOP in FIG. 14B, i.e.GOP1. As shown in FIG. 14A, the stepwise line 1401 shows offset valuesfor the offset sequence with an offset sequence ID equaling 0, i.e.offset sequence [0]. On the other hand, the horizontal line 1402 showsoffset values for the offset sequence with an offset sequence IDequaling 1, i.e. offset sequence [1]. The offset value 1401 of theoffset sequence [0] increases stepwise during the presentation periodGOP1 of the first GOP in the order of frames FR1, FR2, FR3, . . . ,FR15, . . . . As shown in FIG. 14B, the stepwise increase in the offsetvalue 1401 similarly continues in the presentation periods GOP2, GOP3, .. . , GOP40, . . . for the second and subsequent GOPs. The amount ofincrease per frame is sufficiently small for the offset value 1401 inFIG. 14B to appear to increase continually as a line. On the other hand,the offset value 1402 in offset sequence [1] is maintained constantduring the presentation period GOP1 of the first GOP. As shown in FIG.14B, the offset value 1402 increases to a positive value at the end ofthe presentation period GOP40 for the 40^(th) GOP. Offset values maythus exhibit discontinuous change.

FIG. 14C is a schematic diagram showing 3D graphics images reproduced inaccordance with the offset sequences shown in FIGS. 14A and 14B. Whenthe subtitle 3D video image 1403 is displayed in accordance with theoffset sequence [0], the 3D video image 1403 appears to start from rightin front of the screen 1404 and gradually approach the viewer. On theother hand, when the button 3D video image 1405 is displayed inaccordance with the offset sequence [1], the 3D video image 1405 appearsto suddenly jump from a fixed position behind the screen 1404 to infront of the screen 1404. As described, the patterns by which offsetvalues increase and decrease frame by frame are changed in a variety ofways from one offset sequence to another. Individual changes in thedepth of a plurality of 3D graphics images can thereby be represented ina variety of ways.

[Relationship Between Offset Metadata and TS Priority]

FIG. 15 is a schematic diagram showing a PES packet 1510 storing VAU #11500 in the dependent-view video stream and a sequence of TS packets1520 generated from the PES packet 1510. The VAU #1 1500 is located atthe top of the video sequence, like the VAU #1 832 shown in FIG. 8.Accordingly, at least one piece of supplementary data 1504 included inthe VAU #1 1500 consists only of offset metadata 1509. Hatched areas inFIG. 15 show the supplementary data 1504 consisting only of the offsetmetadata 1509. The VAU #1 1500 is stored in the PES payload 1512 of thePES packet 1510. The PES header 1511 of the PES packet 1510 includes aDTS and a PTS assigned to compressed picture data 1505 in the VAU #11500. The PES packet 5010 is stored in the TS packet sequence 5020 inorder from the top. With this arrangement, the TS packet sequence 1520is divided into three groups 1521, 1522, and 1523 in order from the top.The first group 1521 includes the PES header 1511, sub-AU identificationcode 1501, sub-sequence header 1502, and picture header 1503. The secondgroup 1522 only includes the supplementary data 1504 consisting only ofthe offset metadata 1509. The third group 1513 includes the compressedpicture data 1505, padding data 1506, sequence end code 1507, and streamend code 1508. Note that supplementary data other than the offsetmetadata, if included in the VAU #1 1500, is stored in the first group1521 or the third group 1513. The TS packet 1530 located at the end ofthe first group 1521 generally includes an AD field 1532. This preventsthe supplementary data 1504 consisting only of the offset metadata 1509from mixing into the TS payload 1533. Similarly, the TS packet 1550located at the end of the second group 1522 generally includes an ADfield 1552. This prevents the compressed picture data 1505 and any otherdata except the supplementary data 1504 consisting only of the offsetmetadata 1509 from mixing into the TS payload 1553. In this way, thesupplementary data 1504 consisting only of the offset metadata 1509 isstored only into the TS payloads 1542, 1553 of the TS packets 1540, 1550belonging to the second group 1522. On the other hand, the compressedpicture data 1505 is stored only into the TS payloads 1562 of the TSpackets 1560 belonging to the third group 1523.

Further referring to FIG. 15, the TS headers 1531 of the TS packets 1530belonging to the first group 1521 indicate the value of TS priority setto “0”. Similarly, the TS headers 1561 of TS packets 1560 belonging tothe third group 1523 indicate the value of TS priority set to “0”. Onthe other hand, the TS headers 1541, 1551 of the TS packets 1540, 1550belonging to the second group 1522 indicate the value of TS priority setto “1”. Note that these values may be set in reverse. In this way, theTS packets belonging to the second group 1522 have a different value ofTS priority from the TS packets belonging to the other groups 1521 and1523. Accordingly, the system target decoder in the playback device 102can easily select TS packets belonging to the second group 1522 by usingTS priority.

In contrast to FIG. 15, TS packets belonging to the first group 1521 andthe second group 1522 may indicate the same value of TS priority. FIG.16 is a schematic diagram showing a sequence of TS packets 1620 in thatcase. Like the sequence of TS packets 1520 shown in FIG. 15, thesequence of TS packets 1620 is divided into three groups 1621, 1622, and1623 in order from the top. The first group 1621 includes the PES header1511, sub-AU identification code 1501, sub-sequence header 1502, andpicture header 1503. The second group 1622 includes the supplementarydata 1504 consisting only of the offset metadata 1509. The third group1613 includes the compressed picture data 1505, padding data 1506,sequence end code 1507, and stream end code 1508. Note thatsupplementary data other than the offset metadata, if included in theVAU #1 1500, is stored in the first group 1621 or the third group 1613.Hatched areas in FIG. 16 show the supplementary data 1504 consistingonly of the offset metadata 1509, and dotted areas show data 1511,1501-1503 arranged before the supplementary data in the PES packet 1510.The TS headers 1631, 1641, 1651 of the TS packets 1630, 1640, 1650belonging to the first group 1621 and the second group 1622 indicate thevalue of TS priority set to “1”. On the other hand, the TS headers 1661of the TS packets 1660 belonging to the third group 1623 indicate thevalue of TS priority set to “0”. Note that these values may be set inreverse. Even in this case, the TS packets containing the supplementarydata 1504 consisting only of the offset metadata 1509 have a differentvalue of TS priority from the TS packets containing the compressedpicture data 1505. Accordingly, the system target decoder in theplayback device 102 can easily separate the group of TS packetscontaining the supplementary data 1504 consisting only of the offsetmetadata 1509 from the group of TS packets containing the compressedpicture data 1505 by using TS priority.

[Decoding Switch Information]

FIG. 17A is a schematic diagram showing a data structure of decodingswitch information 1750. The decoding switch information 1750 isincluded in the supplementary data in each VAU in both the base-viewvideo stream and the dependent-view video stream shown in FIG. 8.However, in VAU #1 832 located at the top of each VOP in thedependent-view video stream, the decoding switch information 1750 isstored in supplementary data that is different from the supplementarydata 832D containing the offset metadata. The supplementary data 831Dand 832D, in particular in MPEG-4 AVC and MVC, correspond to “SEI” thatis a kind of NAL unit. The decoding switch information 1750 isinformation to cause the decoder in the playback device 102 to easilyspecify the next VAU to decode. As described below, the decoderalternately decodes the base-view video stream and the dependent-viewvideo stream in units of VAUs. When doing so, the decoder generallyspecifies the next VAU to be decoded in alignment with the time shown bythe DTS assigned to each VAU. Many types of decoders, however, continueto decode VAUs in order, ignoring the DTS. For such decoders, it ispreferable for each VAU to include decoding switch information 1750 inaddition to a DTS.

As shown in FIG. 17A, decoding switch information 1750 includes asubsequent access unit type 1751, subsequent access unit size 1752, anddecoding counter 1753. The subsequent access unit type 1751 indicateswhether the next VAU to be decoded belongs to a base-view video streamor a dependent-view video stream. For example, when the value of thesubsequent access unit type 1751 is “1”, the next VAU to be decodedbelongs to a base-view video stream, and when the value of thesubsequent access unit type 1751 is “2”, the next VAU to be decodedbelongs to a dependent-view video stream. When the value of thesubsequent access unit type 1751 is “0”, the current VAU is located atthe end of the stream targeted for decoding, and the next VAU to bedecoded does not exist. The subsequent access unit size 1752 indicatesthe size of the next VAU that is to be decoded. By referring to thesubsequent access unit size 1752, the decoder in the playback device 102can specify the size of a VAU without analyzing its actual structure.Accordingly, the decoder can easily extract VAUs from the buffer. Thedecoding counter 1753 shows the decoding order of the VAU to which itbelongs. The order is counted from a VAU that includes an I picture inthe base-view video stream.

FIG. 17B is a schematic diagram showing sequences of decoding counters1710 and 1720 allocated to each picture in a base-view video stream 1701and a dependent-view video stream 1702. As shown in FIG. 17B, thedecoding counters 1710 and 1720 are incremented alternately between thetwo video streams 1701 and 1702. For example, for VAU 1711 that includesan I picture in the base-view video stream 1701, a value of “1” isassigned to the decoding counter 1710. Next, a value of “2” is assignedto the decoding counter 1720 for the VAU 1721 that includes the next Ppicture to be decoded in the dependent-view video stream 1702.Furthermore, a value of “3” is assigned to the decoding counter 1710 forthe VAU 1712 that includes the next P picture to be decoded in thebase-view video stream 1701. By assigning values in this way, even whenthe decoder in the playback device 102 fails to read one of the VAUs dueto some error, the decoder can immediately specify the missing pictureusing the decoding counters 1710 and 1720. Accordingly, the decoder canperform error processing appropriately and promptly.

In the example shown in FIG. 17B, an error occurs during the reading ofthe third VAU 1713 in the base-view video stream 1701, and the Brpicture is missing. During decoding of the P picture contained in thesecond VAU 1722 in the dependent-view video stream 1702, however, thedecoder has read the decoding counter 1720 for this VAU 1722 andretained the value. Accordingly, the decoder can predict the decodingcounter 1710 for the next VAU to be processed. Specifically, thedecoding counter 1720 in the VAU 1722 that includes the P picture is“4”. Therefore, the decoding counter 1710 for the next VAU to be readcan be predicted to be “5”. The next VAU that is actually read, however,is the fourth VAU 1714 in the base-view video stream 1701, whosedecoding counter 1710 is “7”. The decoder can thus detect that it failedto read a VAU. Accordingly, the decoder can execute the followingprocessing: “skip decoding of the B picture extracted from the third VAU1723 in the dependent-view video stream 1702, since the Br picture to beused as a reference is missing”. In this way, the decoder checks thedecoding counters 1710 and 1720 during each decoding process.Consequently, the decoder can promptly detect errors during reading ofVAUs and can promptly execute appropriate error processing. As a result,the decoder can prevent noise from contaminating the playback video.

FIG. 17C is a schematic diagram showing other examples of the decodingcounters 1730 and 1740 allocated to each picture in a base-view videostream 1701 and a dependent-view video stream 1702. As shown in FIG.12C, decoding counters 1730 and 1740 are incremented separately in thevideo streams 1701 and 1702. Therefore, the decoding counters 1730 and1740 are the same for a pair of pictures in the same 3D VAU. In thiscase, when the decoder has decoded a VAU in the base-view video stream1701, it can predict that “the decoding counter 1230 is the same as thedecoding counter 1740 for the next VAU to be decoded in thedependent-view video stream 1702”. Conversely, when the decoder hasdecoded a VAU in the dependent-view video stream 1702, it can predictthat “the decoding counter 1730 for the next VAU to be decoded in thebase-view video stream 1701 is the same as the decoding counter 1740plus one”. Accordingly, at any point in time, the decoder can promptlydetect an error in reading a VAU using the decoding counters 1730 and1740 and can promptly execute appropriate error processing. As a result,the decoder can prevent noise from contaminating the playback video.

<<Other TS Packets Included in AV Stream File>>

In addition to the TS packets converted from the elementary stream asshown in FIG. 3, the types of TS packets included in an AV stream fileinclude a Program Association Table (PAT), Program Map Table (PMT), andProgram Clock Reference (PCR). The PCR, PMT, and PAT are specified bythe European Digital Broadcasting Standard and are intended to regulatethe partial transport stream constituting a single program. By usingPCR, PMT, and PAT, the AV stream file can also be regulated in the sameway as the partial transport stream. Specifically, the PAT shows the PIDof a PMT included in the same AV stream file. The PID of the PAT itselfis 0. The PMT includes the PIDs for the elementary streams representingvideo, audio, subtitles, etc. included in the same AV stream file, aswell as the attribute information for the elementary streams. The PMTalso includes various descriptors relating to the AV stream file. Thedescriptors particularly include copy control information showingwhether copying of the AV stream file is permitted or not. The PCRincludes information indicating the value of a system time clock (STC)to be associated with the ATS assigned to the PCR itself. The STCreferred to here is a clock used as a reference for the PTS and the DTSby a decoder in the playback device 102. This decoder uses the PCR tosynchronize the STC with the ATC.

FIG. 18 is a schematic diagram showing a data structure of a PMT 1810.The PMT 1810 includes a PMT header 1801, descriptors 1802, and pieces ofstream information 1803. The PMT header 1801 indicates the length ofdata, etc. stored in the PMT 1810. Each descriptor 1802 relates to theentire AV stream file that includes the PMT 1810. The copy controlinformation is included in one of the descriptors 1802. Each piece ofstream information 1803 relates to one of the elementary streamsincluded in the AV stream file and is assigned to a different elementarystream. Each piece of stream information 1803 includes a stream type1831, a PID 1832, and stream descriptors 1833. The stream type 1831includes identification information for the codec used for compressingthe elementary stream. The PID 1832 indicates the PID of the elementarystream. The stream descriptors 1833 include attribute information of theelementary stream, such as a frame rate and an aspect ratio.

By using PCR, PMT, and PAT, the decoder in the playback device 102 canbe made to process the AV stream file in the same way as the partialtransport stream in the European Digital Broadcasting Standard. In thisway, it is possible to ensure compatibility between a playback devicefor the BD-ROM disc 101 and a terminal device conforming to the EuropeanDigital Broadcasting Standard.

<<Interleaved Arrangement of Multiplexed Stream Data>>

For seamless playback of 3D video images, the physical arrangement ofthe base-view video stream and dependent-view video stream on the BD-ROMdisc 101 is important. This “seamless playback” refers to playing backvideo and audio from multiplexed stream data without interruption.

FIG. 19 is a schematic diagram showing a physical arrangement ofmultiplexed stream data on the BD-ROM disc 101. As shown in FIG. 19, themultiplexed stream data is divided into a plurality of data blocks D[n],B[n] (n=0, 1, 2, 3, . . . ) and arranged on the BD-ROM disc 101. A “datablock” refers to a sequence of data recorded on a contiguous area on theBD-ROM disc 101, i.e. a plurality of physically contiguous sectors.Since physical addresses and logical addresses on the BD-ROM disc 101are substantially the same, the LBNs within each data block are alsocontinuous. Accordingly, the BD-ROM drive 121 can continuously read adata block without causing the optical pickup to perform a seek.Hereinafter, data blocks B[n] belonging to a main TS are referred to as“base-view data blocks”, and data blocks D[n] belonging to a sub-TS arereferred to as “dependent-view data blocks”. In particular, data blocksthat include the right-view video stream are referred to as “right-viewdata blocks”, and the data blocks that include the depth map stream arereferred to as “depth map data blocks”.

In the file system on the BD-ROM disc 101, each data block B[n] and D[n]can be accessed as one extent in the files 2D or the files DEP. In otherwords, the logical address for each data block can be known from thefile entry of a file 2D or a file DEP (see <<Supplementary Explanation>>for details).

In the example shown in FIG. 19, the file entry 1910 in the file 2D(01000.m2ts) 241 indicates the sizes of the base-view data blocks B[n]and the LBNs of their tops. Accordingly, the base-view data blocks B[n]can be accessed as extents EXT2D[n] in the file 2D 241. Hereinafter, theextents EXT2D[n] belonging to the file 2D 241 are referred to as “2Dextents”. On the other hand, the file entry 1920 of the file DEP(02000.m2ts) 242 indicates the sizes of the dependent-view data blocksD[n] and the LBNs of their tops. Accordingly, each dependent-view datablock D[n] can be accessed as an extent EXT2[n] in the file DEP 242.Hereinafter, the extents EXT2[n] belonging to the file DEP 242 arereferred to as “dependent-view extents”.

As shown in FIG. 19, a data block group is recorded continuously along atrack on the BD-ROM disc 101. Furthermore, the base-view data blocksB[n] and the dependent-view data blocks D[n] are arranged alternatelyone by one. This type of arrangement of a data block group is referredto as an “interleaved arrangement”. In particular, one series of datablocks recorded in an interleaved arrangement is referred to as an“extent block”. Three extent blocks 1901, 1902, and 1903 are shown inFIG. 19. As shown in the first two extent blocks 1901 and 1902, astorage area NAV for data other than multiplexed stream data existsbetween the extent blocks, thus separating the extent blocks. Also, whenthe BD-ROM disc 101 is a multi-layer disc, i.e. when the BD-ROM disc 101includes a plurality of recording layers, the extent blocks may alsoseparated by a layer boundary LB between the recording layers, as in thesecond and third extent blocks 1902 and 1903 shown in FIG. 19. In thisway, one series of multiplexed stream data is generally arranged so asto be divided into a plurality of extent blocks. In this case, for theplayback device 102 to seamlessly play back video images from themultiplexed stream data, it is necessary for video images to be playedback from the extent blocks to be seamlessly connected. Hereinafter,processing required by the playback device 102 for that purpose isreferred to as “seamless connection between extent blocks”.

The extent blocks 1901-1903 have the same number of the two types ofdata blocks, D[n] and B[n]. Furthermore, the extent ATC time is the samebetween contiguous data block pair D[n] and B[n]. In this context, an“Arrival Time Clock (ATC)” refers to a clock that acts as a standard foran ATS. Also, the “extent ATC time” is defined by the value of the ATCand represents the range of the ATS assigned to source packets in anextent, i.e. the time interval from the ATS of the source packet at thetop of the extent to the ATS of the source packet at the top of the nextextent. In other words, the extent ATC time is the same as the timerequired to transfer all of the source packets in the extent from theread buffer in the playback device 102 to the system target decoder. The“read buffer” is a buffer memory in the playback device 102 where datablocks read from the BD-ROM disc 101 are temporarily stored before beingtransmitted to the system target decoder. Details on the read buffer areprovided later. In the example shown in FIG. 19, since three extentblocks 1901-1903 are connected together seamlessly, the extent ATC timesare the same between the data block pairs D[n], B[n] (n=0, 1, 2, . . .).

The VAUs located at the top of contiguous data blocks D[n] and B[n]belong to the same 3D VAU, and in particular include the top picture ofthe GOP representing the same 3D video image. For example, when thedependent-view data block D[n] is a right-view data block D[n], the topof each right-view data block D[n] includes a P picture for theright-view video stream, and the top of the base-view data block B[n]includes an I picture for the base-view video stream. The P picture forthe right-view video stream represents the right view when the 2D videoimage represented by the I picture in the base-view video stream is usedas the left view. In particular, the P picture, as shown in FIG. 7, iscompressed using the I picture as a reference picture. Accordingly, theplayback device 102 in 3D playback mode can start playback of 3D videoimages from any pair of data blocks D[n] and B[n]. That is to say,processing that requires random access of video streams, such asinterrupt playback, is possible. This holds true also in the case wherethe dependent-view data block D[n] is a depth map data block.

Furthermore, in the interleaved arrangement, among contiguous pairs ofdata blocks D[n] and B[n], dependent-view data blocks D[n] are locatedbefore the base-view data blocks B[n]. This is because generally theamount of data is smaller in the dependent-view data block D[n] than thebase-view data block B[n], i.e. the bit rate is lower. For example, thepicture included in the right-view data block D[n] is compressed usingthe picture included in the base-view data block B[n] as a referencepicture. Accordingly, the size S_(exo)[n] of the right-view data blockD[n] is generally equal to or less than the size S_(EXT1)[n] of thebase-view data block B[n]: S_(EXT2)[n]≦S_(EXT1)[n]. On the other hand,the amount of data per pixel in the depth map, i.e. the number of bitsof the depth value, is in general smaller than the amount of data perpixel of the base-view picture, i.e. the sum of the number of bits ofthe chromatic coordinate value and the a value. Furthermore, as shown inFIGS. 3A and 3B, unlike the sub-TS, the main TS includes otherelementary streams, such as a primary audio stream, in addition to theprimary video stream. Therefore, the size of the depth map data block,S_(EXT3)[n], is generally less than or equal to the size of thebase-view data block B[n], S_(EXT1)[n]: S_(EXT3)[n] 5 S_(EXT1)[n].

[Significance of Dividing Multiplexed Stream Data into Data Blocks]

In order to play 3D video images back seamlessly from the BD-ROM disc101, the playback device 102 has to process the main TS and sub-TS inparallel. The read buffer capacity usable in such processing, however,is generally limited. In particular, there is a limit to the amount ofdata that can be continuously read into the read buffer from the BD-ROMdisc 101. Accordingly, the playback device 102 has to read sections ofthe main TS and sub-TS with the same extent ATC time by dividing thesections.

FIG. 20A is a schematic diagram showing the arrangement of the main TS2001 and sub-TS 2002 recorded separately and consecutively on a BD-ROMdisc. When the playback device 102 processes the main TS 2001 and sub-TS2002 in parallel, as shown by the arrows (1)-(4) on the solid lines inFIG. 20A, the BD-ROM drive 121 alternately reads sections of the main TS2001 and the sub-TS 2002 that have the same extent ATC time. At thistime, as shown by the arrows in the dashed lines in FIG. 20A, duringread processing the BD-ROM drive 121 has to make a large change in thearea to be read on the BD-ROM disc. For example, after the top sectionof the main TS 2001 shown by arrow (1) is read, the BD-ROM drive 121temporarily stops the read operation by the optical pickup and increasesthe rotation speed of the BD-ROM disc. In this way, the BD-ROM drive 121rapidly moves the sector on the BD-ROM disc on which the top section ofthe sub-TS 2002 shown by arrow (2) is recorded to the position of theoptical pickup. This operation to temporarily stop reading by theoptical pickup and, while reading is stopped, position the opticalpickup above the next area to be read is referred to as a “jump”. Thedashed lines with an arrow shown in FIG. 20A indicate the range of thejumps necessary during read processing. During each jump period, readprocessing by the optical pickup stops, and only decoding by the decoderprogresses. Since the jump is excessive in the example shown in FIG.20A, it is difficult to cause read processing to keep up with decoding.As a result, it is difficult to stably maintain seamless playback.

FIG. 20B is a schematic diagram showing an arrangement of dependent-viewdata blocks D[0], D[1], D[2], . . . and base-view data blocks B[0],B[1], B[2], . . . recorded alternately on the BD-ROM disc 101 accordingto Embodiment 1 of the present invention. As shown in FIG. 20B, the mainTS and sub-TS are divided into a plurality of data blocks and arearranged alternately. In this case, during playback of 3D video images,the playback device 102 reads data blocks D[0], B[0], D[1], B[1] . . .in order from the top, as shown by arrows (1)-(4) in FIG. 20B. By simplyreading these data blocks in order, the playback device 102 can smoothlyread the main TS and sub-TS alternately. In particular, since no jumpoccurs during read processing, seamless playback of 3D video images canbe stably maintained.

[Significance of Providing Contiguous Data Blocks with the Same ExtentATC Time]

FIG. 20C is a schematic diagram showing an example of the extent ATCtimes for a dependent-view data block group D[n] and a base-view datablock group B[n] recorded in an interleaved arrangement (n=0, 1, 2). Asshown in FIG. 20C, the extent ATC time is the same in each pair betweenthe dependent-view data block D[n] and the immediately subsequentbase-view data block B[n]. For example, the extent ATC time is equal toone second for each of D[0] and B[0] in the top data block pair.Accordingly, when the data blocks D[0] and B[0] are read by the readbuffer in the playback device 102, all of the TS packets therein aresent from the read buffer to the system target decoder in the sameone-second interval. Similarly, since the extent ATC time is equal to0.7 seconds for each of D[1] and B[1] in the second data block pair, allof the TS packets in each data block are transmitted from the readbuffer to the system target decoder in the same 0.7-second interval.

FIG. 20D is a schematic diagram showing another example of the extentATC times for a dependent-view data block group D[n] and a base-viewdata block group B[n] recorded in an interleaved arrangement. As shownin FIG. 20D, the extent ATC times in all of the data blocks D[n] andB[n] are equal to one second. Accordingly, in the same one-secondinterval in which any of the data blocks D[n] and B[n] are read by theread buffer in the playback device 102, all of the TS packets in each ofthose data blocks are transmitted from the read buffer to the systemtarget decoder.

As described above, the compression rate of the dependent-view datablocks is generally higher than the compression rate of the base-viewdata blocks. Accordingly, decoding of the dependent-view data blocks isgenerally slower than decoding of the base-view data blocks. On theother hand, when the extent ATC times are equal, the dependent-view datablocks have a smaller amount of data than the base-view data blocks.Therefore, when the extent ATC times are the same between contiguousdata blocks as in FIGS. 20C and 20D, the speed at which the data to bedecoded is provided to the system target decoder can easily bemaintained uniformly with the speed of processing by the decoder. Inother words, the system target decoder facilitates synchronizationbetween the decoding of the base-view data blocks and the decoding ofthe dependent-view data blocks, particularly in interrupt playback.

[Significance of Placing Smaller-Data-Amount Data Blocks First]

When reading a data block located at the top or at the playback startposition of each extent block, the playback device 102 in 3D playbackmode first reads the entirety of the data block into the read buffer.The data block is not transferred to the system target decoder duringthat period. After finishing reading the data block, the playback device102 transfers the data block to the system target decoder in parallelwith the next data block. This processing is called “preloading”.

The technical significance of preloading is as follows. First, in L/Rmode, base-view data blocks are necessary for decoding thedependent-view data blocks. Therefore, to maintain the buffer at theminimum necessary capacity for storing the decoded data until outputprocessing, it is preferable to simultaneously provide the data blocksto the system target decoder to be decoded. On the other hand, in depthmode, processing is necessary to generate a pair of video planesrepresenting parallax images from a pair of a decoded base-view pictureand a decoded depth map. Accordingly, to maintain the buffer at theminimum necessary capacity for storing the decoded data until thisprocessing, it is preferable to provide the base-view data blockssimultaneously with the depth map data blocks to the system targetdecoder to be decoded. Therefore, preloading causes the entirety of thedata block at the top of an extent block or at the playback startposition to be read into the read buffer in advance. This enables thedata block and the following data block to be transferred simultaneouslyfrom the read buffer to the system target decoder and decoded.Furthermore, the subsequent pairs of data blocks can also besimultaneously decoded by the system target decoder.

When preloading, the entirety of the data block that is read first isstored in the read buffer. Accordingly, the read buffer requires atleast a capacity equal to the size of the data block. To maintain thecapacity of the read buffer at a minimum, the size of the data block tobe preloaded should be as small as possible. Meanwhile, for interruptplayback, etc., any pair of data blocks may be selected as the playbackstart position. For this reason, the data block having the smallest dataamount is placed first in each pair of the data blocks. This enables theminimum capacity to be maintained in the read buffer.

<<Cross-Linking of AV Stream Files to Data Blocks>>

For the data block group shown in FIG. 19, the AV stream files arecross-linked as follows. The file entry 1940 of the file SS (01000.ssif)244A considers each extent block 1901-1903 to each be one extent,indicating the size of each and the LBN of the top thereof. Accordingly,the extent blocks 1901-1903 can be accessed as the extents EXTSS[0],EXTSS[1], and EXTSS[2] of the file SS 244A. Hereinafter, the extentsEXTSS[0], EXTSS[1], and EXTSS[2] belonging to the file SS 244A arereferred to as the “extents SS”. Each of the extents SS EXTSS[0],EXTSS[1], and EXTSS[2] share the base-view data blocks B[n] with thefile 2D 241 and share the dependent-view data blocks D[n] with the fileDEP 242.

<<Playback Path for Extent Block Group>>

FIG. 21 is a schematic diagram showing a playback path 2101 in 2Dplayback mode for an extent block group 1901-1903. The playback device102 in 2D playback mode plays back the file 2D 241. Accordingly, asindicated by the playback path 2101 in 2D playback mode, the base-viewdata blocks B[n] (n=0, 1, 2, . . . ) are read in order from the extentblocks 1901-1903 as 2D extents EXT2D[0], EXT2D[1], and EXT2D[2].Specifically, first, the top base-view data block B[0] is read from thetop extent block 1901, then reading of the immediately subsequentdependent-view data block D[0] is skipped by a first jump J_(2D) 1.Next, the second base-view data block B[1] is read, and then reading ofthe immediately subsequent data NAV and dependent-view data block D[1]is skipped by a second jump J_(NAV). Subsequently, reading of thebase-view data blocks and jumps are repeated similarly in the second andsubsequent extent blocks 1902 and 1903.

A jump J_(LY) occurring between the second extent block 1902 and thethird extent block 1903 is a long jump across the layer boundary LB. A“long jump” is a collective term for jumps with a long seek time andspecifically refers to a jump distance that exceeds a predeterminedthreshold value. “Jump distance” refers to the length of the area on theBD-ROM disc 101 whose reading is skipped during a jump period. Jumpdistance is normally expressed as the number of sectors of thecorresponding section. The threshold value used to define a long jump isspecified, for example, as 40000 sectors in the BD-ROM standard. Thisthreshold value, however, depends on the type of BD-ROM disc and on theBD-ROM drive's read processing capability. Long jumps particularlyinclude focus jumps and track jumps. A “focus jump” is a jump caused byswitching recording layers, and includes processing to change the focusdistance of the optical pickup. A “track jump” includes processing tomove the optical pickup in a radial direction along the BD-ROM disc 101.

FIG. 21 further shows a playback path 2102 in L/R mode for the extentblock group 1901-1903. The playback device 102 in L/R mode plays backthe file SS 244A. Accordingly, as indicated by the playback path 2102 inL/R mode, the extent blocks 1901-1903 are read in order as the extentsSS EXTSS[0], EXTSS[1], and EXTSS[2]. Specifically, the data blocks D[0],B[0], D[1] and B[1] are first sequentially read from the top extentblock 1901, then reading of the immediately subsequent data NAV isskipped by a first jump J_(NAV). Next, the data blocks D[2], . . . ,B[3] are sequentially read from the second extent block 1902.Immediately thereafter, a long jump J_(LY) occurs at the same time asswitching the recording layer, and next, the data blocks D[4], B[4], . .. are sequentially read from the third extent block 1903.

When reading the extent blocks 1901-1903 as extents of the file SS 244A,the playback device 102 reads the top LBN of the extents SS EXTSS[0],EXTSS[1], . . . and the size thereof, from the file entry 1940 in thefile SS 244A and then outputs the LBNs and sizes to the BD-ROM drive121. The BD-ROM drive 121 continuously reads data having the input sizefrom the input LBN. In such processing, control of the BD-ROM drive 121is easier than processing to read the data block groups as the extentsin the first file DEP 242 and the file 2D 241 for the following reasons(A) and (B): (A) the playback device 102 may refer in order to extentsusing a file entry in one location, and (B) since the total number ofextents to be read substantially halves, the total number of pairs of anLBN and a size that need to be output to the BD-ROM drive 121 halves.However, after the playback device 102 has read the extents SS EXTSS[0],EXTSS[1], . . . , it needs to separate each into a dependent-view datablock and a base-view data block and output them to the decoder. Theclip information file is used for this separation processing. Detailsare provided below.

As shown in FIG. 19, when actually reading the extent blocks 1901-1903,the BD-ROM drive 121 performs a zero sector transition J₀ in the timefrom the top of a data block to the top of the next data block. A “zerosector transition” is a movement of the optical pickup between twoconsecutive data blocks. During a period in which a zero sectortransition is performed (hereinafter referred to as a “zero sectortransition period”), the optical pickup temporarily suspends its readoperation and waits. In this sense, the zero sector transition isconsidered “a jump in which the jump distance is equal to 0 sectors”.The length of the zero sector transition period, that is, the zerosector transition time period, may include, in addition to the time forshifting the position of the optical pickup via revolution of the BD-ROMdisc 101, overhead caused by error correction processing. “Overheadcaused by error correction processing” refers to excess time caused byperforming error correction processing twice using an ECC block when theboundary between ECC blocks does not match the boundary between two datablocks. A whole ECC block is necessary for error correction processing.Accordingly, when two consecutive data blocks share a single ECC block,the whole ECC block is read and used for error correction processingduring reading of either data block. As a result, each time one of thesedata blocks is read, a maximum of 32 sectors of excess data isadditionally read. The overhead caused by error correction processing isassessed as the total time for reading the excess data, i.e. 32sectors×2048 bytes×8 bits/byte×2 instances/read rate. Note that byconfiguring each data block in ECC block units, the overhead caused byerror correction processing may be removed from the zero sectortransition time.

<<Clip Information File>>

FIG. 22 is a schematic diagram showing a data structure of a first clipinformation file (01000.clpi), i.e. the 2D clip information file 231.The dependent-view clip information file (02000.clip) 232 and the clipinformation file (03000.clpi) 233 have the same data structure. Below,the data structure common to all clip information files is described,first using the data structure of the 2D clip information file 231 as anexample. Afterwards, the differences in data structure between a 2D clipinformation file and a dependent-view clip information file aredescribed.

As shown in FIG. 22, the 2D clip information file 231 includes clipinformation 2210, stream attribute information 2220, an entry map 2230,and 3D metadata 2240. The 3D metadata 2240 includes extent start points2242.

The clip information 2210 includes a system rate 2211, a playback starttime 2212, and a playback end time 2213. The system rate 2211 specifiesa system rate for the file 2D (01000.m2ts) 241. The playback device 102in 2D playback mode transfers TS packets belonging to the file 2D 241from the read buffer to the system target decoder. The “system rate”refers to the upper limit of the transfer rate. The interval between theATSs of the source packets in the file 2D 241 is set so that thetransfer speed is limited to the system rate or lower. The playbackstart time 2212 indicates the PTS of the VAU located at the top of thefile 2D 241, e.g. the PTS of the top video frame. The playback end time2212 indicates the value of the STC delayed a predetermined time fromthe PTS of the VAU located at the end of the file 2D 241, e.g. the sumof the PTS of the last video frame and the playback time of one frame.

The stream attribute information 2220 is a correspondence table betweenthe PID 2221 for each elementary stream included in the file 2D 241 andpieces of attribute information 2222. Each piece of attributeinformation 2222 is different for a video stream, audio stream, PGstream, and IG stream. For example, the attribute informationcorresponding to the PID 0x1011 for the primary video stream includes acodec type used for the compression of the video stream, as well as aresolution, aspect ratio, and frame rate for each picture constitutingthe video stream. On the other hand, the attribute informationcorresponding to the PID 0x1100 for the primary audio stream includes acodec type used for compressing the audio stream, a number of channelsincluded in the audio stream, language, and sampling frequency. Theplayback device 102 uses this attribute information 2222 to initializethe decoder.

[Entry Map]

FIG. 23A is a schematic diagram showing a data structure of an entry map2230. As shown in FIG. 23A, the entry map 2230 includes tables 2300.There is the same number of tables 2300 as there are video streamsmultiplexed in the main TS, and tables are assigned one-by-one to eachvideo stream. In FIG. 23A, each table 2300 is distinguished by the PIDof the video stream to which it is assigned. Each table 2300 includes anentry map header 2301 and an entry point 2302. The entry map header 2301includes the PID corresponding to the table 2300 and the total number ofentry points 2302 included in the table 2300. An entry point 2302associates each pair of a PTS 2303 and source packet number (SPN) 2304with one of individually differing entry points ID (EP_ID) 2305. The PTS2303 is equivalent to the PTS for one of the I pictures included in thevideo stream for the PID indicated by the entry map header 2301. The SPN2304 is equivalent to the SPN for the top of the source packet groupstored in the corresponding I picture. An “SPN” refers to the serialnumber assigned consecutively from the top to a source packet groupbelonging to one AV stream file. The SPN is used as the address for eachsource packet in the AV stream file. In the entry map 2230 in the 2Dclip information file 231, the SPN refers to the number assigned to thesource packet group belonging to the file 2D 241, i.e. the source packetgroup constituting the main TS. Accordingly, the entry point 2302expresses the correspondence between the PTS and the address, i.e. theSPN, of each I picture included in the file 2D 241.

An entry point 2302 does not need to be set for all of the I pictures inthe file 2D 241. However, when an I picture is located at the top of aGOP, and the TS packet that includes the top of that I picture islocated at the top of a 2D extent, an entry point 2302 has to be set forthat I picture.

FIG. 23B is a schematic diagram showing source packets in a sourcepacket group 2310 belonging to a file 2D 241 that are associated witheach EP_ID 2305 by the entry map 2230. FIG. 23C is a schematic diagramshowing a data block group D[n], B[n] (n=0, 1, 2, 3, . . . ) on a BD-ROMdisc 101 corresponding to the source packet group 2310. When theplayback device 102 plays back 2D video images from the file 2D 241, itrefers to the entry map 2230 to specify the SPN for the source packetthat includes a frame representing an arbitrary scene from the PTS forthat frame. Specifically, when for example a PTS=360000 is indicated asthe PTS for a specific entry point for the playback start position, theplayback device 102 first retrieves the SPN=3200 allocated to this PTSin the entry map 2230. Next, the playback device 102 seeks the quotientSPN×192/2048, i.e. the value of the SPN multiplied by 192 bytes, thedata amount per source packet, and divided by 2048 bytes, the dataamount per sector. As can be understood from FIGS. 5B and 5C, this valueis the same as the total number of sectors recorded in the main TS priorto the source packet to which the SPN is assigned. In the example shownin FIG. 23B, this value is 3200×192/2048=300, and is equal to the totalnumber of sectors on which source packet groups 2311 are recorded fromSPN 0 through 3199. Next, the playback device 102 refers to the fileentry in the file 2D 241 and specifies the LBN of the (totalnumber+1)^(th) sector, counting from the top of the sector groups inwhich 2D extent groups are recorded. In the example shown in FIG. 23C,within the sector groups in which the base-view data blocks B[0], B[1],B[2], . . . which can be accessed as 2D extents EXT2D[0], EXT2D[1],EXT2D[2], . . . are recorded, the LBN of the 301^(st) sector countingfrom the top is specified. The playback device 102 indicates this LBN tothe BD-ROM drive 121. In this way, base-view data block groups are readas aligned units in order from the sector for this LBN. Furthermore,from the first aligned unit that is read in, the playback device 102selects the source packet indicated by the entry point for the playbackstart position and then extracts and decodes an I picture. From then on,subsequent pictures are decoded in order referring to already decodedpictures. In this way, the playback device 102 can play back 2D videoimages from the file 2D 241 from a specified PTS onwards.

Furthermore, the entry map 2230 is useful for efficient processingduring trickplay such as fast forward, reverse, etc. For example, theplayback device 102 in 2D playback mode first refers to the entry map2230 to read SPNs starting at the playback start position, e.g. to readSPN=3200, 4800, . . . in order from the entry points EP_ID=2, 3, . . .that include PTSs starting at PTS=360000. Next, the playback device 102refers to the file entry in the file 2D 241 to specify the LBN of thesectors corresponding to each SPN. The playback device 102 thenindicates each LBN to the BD-ROM drive 121. Aligned units are thus readfrom the sector for each LBN. Furthermore, from each aligned unit, theplayback device 102 selects the source packet indicated by each entrypoint and then extracts and decodes an I picture. The playback device102 can thus selectively play back an I picture from the file 2D 241without analyzing the 2D extent group EXT2D[n] itself.

[Extent Start Point]

FIG. 24A is a schematic diagram showing a data structure of extent startpoints 2242. As shown in FIG. 24A, an “extent start point” 2242 includesbase-view extent IDs (EXT1_ID) 2411 and SPNs 2412. The EXT1_IDs 2411 areserial numbers assigned consecutively from the top to the base-view datablocks belonging to the file SS (01000.ssif) 244A. One SPN 2412 isassigned to each EXT1_ID 2411 and is the same as the SPN for the sourcepacket located at the top of the base-view data block identified by theEXT1_ID 2411. This SPN is a serial number assigned from the top to thesource packets included in the base-view data block group belonging tothe file SS 244A.

In the extent blocks 1901-1903 shown in FIG. 19, the file 2D 241 and thefile SS 244A share the base-view data blocks B[0], B[1], B[2], . . . incommon. However, data block groups placed at locations requiring a longjump, such as at boundaries between recording layers, generally includebase-view data blocks belonging to only one of the file 2D 241 or thefile SS 244A (see <<Supplementary Explanation>> for details).Accordingly, the SPN 2412 that indicates the extent start point 2242generally differs from the SPN for the source packet located at the topof the 2D extent belonging to the file 2D 241.

FIG. 24B is a schematic diagram showing a data structure of extent startpoints 2420 included in a second clip information file (02000.clpi),i.e. the dependent-view clip information file 232. As shown in FIG. 24B,the extent start point 2420 includes dependent-view extent IDs (EXT2_ID)2421 and SPNs 2422. The EXT2_IDs 2421 are serial numbers assigned fromthe top to the dependent-view data blocks belonging to the file SS 244A.One SPN 2422 is assigned to each EXT2_ID 2421 and is the same as the SPNfor the source packet located at the top of the dependent-view datablock identified by the EXT2_ID 2421. This SPN is a serial numberassigned in order from the top to the source packets included in thedependent-view data block group belonging to the file SS 244A.

FIG. 24D is a schematic diagram representing correspondence betweendependent-view extents EXT2[0], EXT2[1], . . . belonging to the file DEP(02000.m2ts) 242 and the SPNs 2422 shown by the extent start points2420. As shown in FIG. 19, the file DEP 242 and the file SS 244A sharedependent-view data blocks in common. Accordingly, as shown in FIG. 24D,each SPN 2422 shown by the extent start points 2420 is the same as theSPN for the source packet located at the top of each dependent-viewextent EXT2[0], EXT2[1], . . . .

As described below, the extent start point 2242 in the 2D clipinformation file 231 and the extent start point 2420 in thedependent-view clip information file 232 are used to detect the boundaryof data blocks included in each extent SS during playback of 3D videoimages from the file SS 244A.

FIG. 24E is a schematic diagram showing an example of correspondencebetween an extent SS EXTSS[0] belonging to the file SS 244A and anextent block on the BD-ROM disc 101. As shown in FIG. 24E, the extentblock includes data block groups D[n] and B[n] (n=0, 1, 2, . . . ) in aninterleaved arrangement. Note that the following description is alsotrue for other arrangements. The extent block can be accessed as asingle extent SS EXTSS[0]. Furthermore, in the extent SS EXTSS[0], thenumber of source packets included in the base-view data block B[n] is,at the extent start point 2242, the same as the difference A(n+1)—Anbetween SPNs corresponding to EXT1_ID=n+1 and n. In this case, A0=0. Onthe other hand, the number of source packets included in thedependent-view data block D[n+1] is, in the extent start point 2420, thesame as the difference B(n+1)−Bn between SPNs corresponding toEXT2_ID=n+1 and n. In this case, B0=0.

When the playback device 102 in 3D playback mode plays back 3D videoimages from the file SS 244A, the playback device 102 refers to theentry maps and the extent start points 2242 and 2420 respectively foundin the clip information files 231 and 232. By doing this, the playbackdevice 102 specifies, from the PTS for a frame representing the rightview of an arbitrary scene, the LBN for the sector on which adependent-view data block that is required as a constituent of the frameis recorded. Specifically, the playback device 102 for example firstretrieves the SPN associated with the PTS from the entry map in thedependent-view clip information file 232. It is assumed that the sourcepacket indicated by the SPN is included in the third dependent-viewextent EXT2[2] in the first file DEP 242, i.e. the dependent-view datablock D[2]. Next, the playback device 102 retrieves “B2”, the largestSPN before the target SPN, from among the SPNs 2422 shown by the extentstart points 2420 in the dependent-view clip information file 232. Theplayback device 102 also retrieves the corresponding EXT2_ID “2”. Thenthe playback device 102 retrieves the value “A2” for the SPN 2412corresponding to the EXT1_ID, which is the same as the EXT2_ID “2”, fromthe extent start points 2242 in the 2D clip information file 231. Theplayback device 102 further seeks the sum B2+A2 of the retrieved SPNs.As can be seen from FIG. 24E, this sum B2+A2 is the same as the totalnumber of source packets included in the data blocks located before thethird dependent-view data block D[2] among the data blocks included inthe extent SS EXTSS[0]. Accordingly, this sum B2+A2 multiplied by 192bytes, the data amount per source packet, and divided by 2048 bytes, thedata amount per sector, i.e. (B2+A2)×192/2048, is the same as the numberof sectors from the top of the extent SS EXTSS[0] until immediatelybefore the third dependent-view data block D[2]. Using this quotient,the LBN for the sector on which the top of the dependent-view data blockD[2] is recorded can be specified by referencing the file entry for thefile SS 244A.

After specifying the LBN via the above-described procedure, the playbackdevice 102 indicates the LBN to the BD-ROM drive 121. In this way, theportion of the extent SS EXTSS[0] recorded starting with the sector forthis LBN, i.e. the data block group D[2], B[2], D[3], B[3], . . .starting from the third dependent-view data block D[2], is read asaligned units.

The playback device 102 further refers to the extent start points 2242and 2420 to extract dependent-view data blocks and base-view data blocksalternately from the read extents SS. For example, assume that the datablock group D[n], B[n] (n=0, 1, 2, . . . ) is read in order from theextent SS EXTSS[0] shown in FIG. 24E. The playback device 102 firstextracts B1 source packets from the top of the extent SS EXTSS[0] as thedependent-view data block D[0]. Next, the playback device 102 extractsthe B1 ^(th) source packet and the subsequent (A1−1) source packets, atotal of A1 source packets, as the first base-view data block B[0]. Theplayback device 102 then extracts the (B1+A1)^(th) source packet and thesubsequent (B2−B1−1) source packets, a total of (B2−B1) source packets,as the second dependent-view data block D[1]. The playback device 102further extracts the (A1+B2)^(th) source packet and the subsequent(A2−A1−1) source packets, a total of (A2−A1) source packets, as thesecond base-view data block B[1]. Thereafter, the playback device 102thus continues to detect the boundary between data blocks in the extentSS based on the number of read source packets, thereby alternatelyextracting dependent-view and base-view data blocks. The extractedbase-view and dependent-view data blocks are transmitted to the systemtarget decoder to be decoded in parallel.

In this way, the playback device 102 in 3D playback mode can play back3D video images from the file SS 244A starting at a specific PTS. As aresult, the playback device 102 can in fact benefit from theabove-described advantages (A) and (B) regarding control of the BD-ROMdrive 121.

<<File Base>>

FIG. 24C is a schematic diagram representing the base-view data blocksB[0], B[1], B[2], . . . extracted from the file SS 244A by the playbackdevice 102 in 3D playback mode. As shown in FIG. 24C, when allocatingSPNs in order from the top to a source packet group included in thebase-view data block B[n] (n=0, 1, 2, . . . ), the SPN of the sourcepacket located at the top of the data block B[n] is equal to the SPN2412 indicating the extent start point 2242. The base-view data blockgroup extracted from a single file SS by referring to extent startpoints, like the base-view data block group B[n], is referred to as a“file base”. Furthermore, the base-view data blocks included in a filebase are referred to as “base-view extents”. As shown in FIG. 24E, eachbase-view extent EXT1[0], EXT1[1] . . . is referred to by an extentstart point 2242 or 2420 in a clip information file.

A base-view extent EXT1[n] shares the same base-view data block B[n]with a

-   -   2D extent EXT2D[n]. Accordingly, the file base includes the same        main TS as the file 2D. Unlike the 2D extent EXT2D[n], however,        the base-view extent EXT1[n] is not referred to by any file        entry. As described above, the base-view extent EXT1[n] is        extracted from the extent SS EXTSS[•] in the file SS with use of        the extent start point in the clip information file. The file        base thus differs from a conventional file by not including a        file entry and by needing an extent start point as a reference        for a base-view extent. In this sense, the file base is a        “virtual file”. In particular, the file base is not recognized        by the file system and does not appear in the directory/file        structure shown in FIG. 2.

FIG. 25 is a schematic diagram showing correspondence between a singleextent block 2500 recorded on the BD-ROM disc 101 and each of the extentblock groups in a file 2D 2510, file base 2511, file DEP 2512, and fileSS 2520. As shown in FIG. 25, the extent block 2500 includes thedependent-view data blocks D[n] and the base-view data blocks B[n] (n= .. . , 0, 1, 2, 3, . . . ). The base-view data block B[n] belongs to thefile 2D 2510 as the 2D extent EXT2D[n]. The dependent-view data blockD[n] belongs to the file DEP 2512 as the dependent-view extent EXT2[n].The entirety of the extent block 2500 belongs to the file SS 2520 as oneextent SS EXTSS[0]. Accordingly, the extent SS EXTSS[0] shares thebase-view data block B[n] in common with the 2D extent EXT2D[n] andshares the dependent-view data block D[n] with the dependent-view extentEXT2[n]. After being read into the playback device 102, the extent SSEXTSS[0] is separated into the dependent-view data block D[n] and thebase-view data block B[n]. These base-view data blocks B[n] belong tothe file base 2511 as the base-view extents EXT1[n]. The boundary in theextent SS EXTSS[0] between the base-view extent EXT1[n] and thedependent-view extent EXT2[n] is specified with use of the extent startpoint in the clip information file corresponding to each of the file 2D2510 and the file DEP 2512.

<<Dependent-View Clip Information File>>

The dependent-view clip information file has the same data structure asthe 2D clip information file shown in FIGS. 22-24. Accordingly, thefollowing description covers the differences between the dependent-viewclip information file and the 2D clip information file. Details on thesimilarities can be found in the above description.

A dependent-view clip information file differs from a 2D clipinformation file mainly in the following two points: (i) conditions areplaced on the stream attribute information, and (ii) conditions areplaced on the entry points.

(i) When the base-view video stream and the dependent-view video streamare to be used for playback of 3D video images by the playback device102 in L/R mode, as shown in FIG. 7, the dependent-view video stream iscompressed using the base-view video stream. At this point, the videostream attributes of the dependent-view video stream become equivalentto the base-view video stream. The video stream attribute informationfor the base-view video stream is associated with PID=0x1011 in thestream attribute information 2220 in the 2D clip information file. Onthe other hand, the video stream attribute information for thedependent-view video stream is associated with PID=0x1012 or 0x1013 inthe stream attribute information in the dependent-view clip informationfile. Accordingly, the items shown in FIG. 22, i.e. the codec,resolution, aspect ratio, and frame rate, have to match between thesetwo pieces of video stream attribute information. If the codec typematches, then a reference relationship between pictures in the base-viewvideo stream and the dependent-view video stream is established duringcoding, and thus each picture can be decoded. If the resolution, aspectratio, and frame rate all match, then on-screen display of the left andright videos can be synchronized. Therefore, these videos can be shownas 3D video images without making the viewer feel uncomfortable.

(ii) The entry map in the dependent-view clip information file includesa table allocated to the dependent-view video stream. Like the table2300 shown in FIG. 23A, this table includes an entry map header andentry points. The entry map header indicates the PID for thedependent-view video stream allocated to the table, i.e. either 0x1012or 0x1013. In each entry point, a pair of a PTS and an SPN is associatedwith a single EP_ID. The PTS for each entry point is the same as the PTSfor the top picture in one of the GOPs included in the dependent-viewvideo stream. The SPN for each entry point is the same as the top SPN ofthe source packet group stored in the picture indicated by the PTSbelonging to the same entry point. This SPN refers to a serial numberassigned consecutively from the top to the source packet group belongingto the file DEP, i.e. the source packet group constituting the sub-TS.The PTS for each entry point has to match the PTS, within the entry mapin the 2D clip information file, for the entry point in the tableallotted to the base-view video stream. In other words, whenever anentry point is set at the top of a source packet group that includes oneof a set of pictures included in the same 3D VAU, an entry point alwayshas to be set at the top of the source packet group that includes theother picture.

FIG. 26 is a schematic diagram showing an example of entry points set ina base-view video stream 2610 and a dependent-view video stream 2620. Inthe two video streams 2610 and 2620, GOPs that are the same number fromthe top represent video for the same playback period. As shown in FIG.26, in the base-view video stream 2610, entry points 2601B, 2603B, and2605B are set at the top of the odd-numbered GOPs as counted from thetop, i.e. GOP #1, GOP #3, and GOP #5. Accordingly, in the dependent-viewvideo stream 2620 as well, entry points 2601D, 2603D, and 2605D are setat the top of the odd-numbered GOPs as counted from the top, i.e. GOP#1, GOP #3, and GOP #5. In this case, when the playback device 102begins playback of 3D video images from GOP #3, for example, it canimmediately calculate the address of the playback start position in thefile SS from the SPN of the corresponding entry points 2603B and 2603D.In particular, when both entry points 2603B and 2603D are set at the topof a data block, then as can be understood from FIG. 24E, the sum of theSPNs of the entry points 2603B and 2603D equals the SPN of the playbackstart position within the file SS. As described with reference to FIG.24E, from this number of source packets, it is possible to calculate theLBN of the sector on which the part of the file SS for the playbackstart position is recorded. In this way, even during playback of 3Dvideo images, it is possible to improve response speed for processingthat requires random access to the video stream, such as interruptplayback or the like.

<<2D Playlist File>>

FIG. 27 is a schematic diagram showing a data structure of a 2D playlistfile. The first playlist file (00001.mpls) 221 shown in FIG. 2 has thisdata structure. As shown in FIG. 27, the 2D playlist file 221 includes amain path 2701 and two sub-paths 2702 and 2703.

The main path 2701 is a sequence of playitem information pieces (PI)that defines the main playback path for the file 2D 241, i.e. thesection for playback and the section's playback order. Each PI isidentified with a unique playitem ID=#N (N=1, 2, 3, . . . ). Each PI #Ndefines a different playback section along the main playback path with apair of PTSs. One of the PTSs in the pair represents the start time(In-Time) of the playback section, and the other represents the end time(Out-Time). Furthermore, the order of the PIs in the main path 2701represents the order of corresponding playback sections in the playbackpath.

Each of the sub-paths 2702 and 2703 is a sequence of sub-playiteminformation pieces (SUB_PI) that defines a playback path that can beassociated in parallel with the main playback path for the file 2D 241.Such a playback path is a different section of the file 2D 241 than isrepresented by the main path 2701, or is a section of stream datamultiplexed in another file 2D, along with the corresponding playbackorder. The playback path may also indicate stream data multiplexed in adifferent file 2D than the file 2D 241 as a section for playback, alongwith the corresponding playback order. The stream data indicated by theplayback path represents other 2D video images to be played backsimultaneously with 2D video images played back from the file 2D 241 inaccordance with the main path 2701. These other 2D video images include,for example, sub-video in a picture-in-picture format, a browser window,a pop-up menu, or subtitles. Serial numbers “0” and “1” are assigned tothe sub-paths 2702 and 2703 in the order of registration in the 2Dplaylist file 221. These serial numbers are used as sub-path IDs toidentify the sub-paths 2702 and 2703. In the sub-paths 2702 and 2703,each SUB_PI is identified by a unique sub-playitem ID=#M (M=1, 2, 3, . .. ). Each SUB_PI #M defines a different playback section along theplayback path with a pair of PTSs. One of the PTSs in the pairrepresents the playback start time of the playback section, and theother represents the playback end time. Furthermore, the order of theSUB_PIs in the sub-paths 2702 and 2703 represents the order ofcorresponding playback sections in the playback path.

FIG. 28 is a schematic diagram showing a data structure of PI #N. Asshown in FIG. 28, a PI #N includes a piece of reference clip information2801, playback start time (In_Time) 2802, playback end time (Out_Time)2803, connection condition 2804, and stream selection table (hereinafterreferred to as “STN table” (stream number table)) 2805. The referenceclip information 2801 is information for identifying the 2D clipinformation file 231. The playback start time 2802 and playback end time2803 respectively indicate PTSs for the beginning and the end of thesection for playback of the file 2D 241. The connection condition 2804specifies a condition for connecting video in the playback sectionspecified by a playback start time 2802 and a playback end time 2803 tovideo in the playback section specified by the previous PI #(N−1). TheSTN table 2805 is a list of elementary streams that can be selected fromthe file 2D 241 by the decoder in the playback device 102 from theplayback start time 2802 until the playback end time 2803.

The data structure of a SUB_PI is the same as the data structure of thePI shown in FIG. 28 insofar as it includes reference clip information, aplayback start time, and a playback end time. In particular, theplayback start time and playback end time of a SUB_PI are expressed asvalues along the same time axis as a PI. The SUB_PI further includes an“SP connection condition” field. The SP connection condition has thesame meaning as a PI connection condition.

[Connection Condition]

The connection condition (hereinafter abbreviated as “CC”) 2804 can forexample be assigned three types of values, “1”, “5”, and “6”. When theCC 2804 is “1”, the video to be played back from the section of the file2D 241 specified by the PI #N does not need to be seamlessly connectedto the video played back from the section of the file 2D 241 specifiedby the immediately preceding PI #(N -1). On the other hand, when the CC2804 indicates “5” or “6”, both video images need to be seamlesslyconnected.

FIGS. 29A and 29B are schematic diagrams showing correspondence betweentwo playback sections 2901 and 2902 that are to be connected when CC is“5” or “6”. In this case, the PI #(N -1) specifies a first section 2901in the file 2D 241, and the PI #N specifies a second section 2902 in thefile 2D 241. As shown in FIG. 29A, when the CC indicates “5”, the STCsof the two PIs, PI #(N−1) and PI #N, may be nonconsecutive. That is, thePTS #1 at the end of the first section 2901 and the PTS #2 at the top ofthe second section 2902 may be nonconsecutive. Several constraintconditions, however, need to be satisfied. For example, the firstsection 2901 and second section 2902 need to be created so that thedecoder can smoothly continue to decode data even when the secondsection 2902 is supplied to the decoder consecutively after the firstsection 2901. Furthermore, the last frame of the audio stream containedin the first section 2901 needs to overlap the top frame of the audiostream contained in the second section 2902. On the other hand, as shownin FIG. 29B, when the CC indicates “6”, the first section 2901 and thesecond section 2902 need to be able to be handled as successive sectionsfor the decoder to duly decode. That is, STCs and ATCs need to becontiguous between the first section 2901 and the second section 2902.Similarly, when the SP connection condition is “5” or “6”, STCs and ATCsboth need to be contiguous between sections of the file 2D specified bytwo contiguous SUB_PIs.

[STN Table]

Referring again to FIG. 28, the STN table 2805 is an array of streamregistration information. “Stream registration information” isinformation individually listing the elementary streams that can beselected for playback from the main TS between the playback start time2802 and playback end time 2803. The stream number (STN) 2806 is aserial number allocated individually to stream registration informationand is used by the playback device 102 to identify each elementarystream. The STN 2806 further indicates priority for selection amongelementary streams of the same type. The stream registration informationincludes a stream entry 2809 and stream attribute information 2810. Thestream entry 2809 includes stream path information 2807 and streamidentification information 2808. The stream path information 2807 isinformation indicating the file 2D to which the selected elementarystream belongs. For example, if the stream path information 2807indicates “main path”, the file 2D corresponds to the 2D clipinformation file indicated by reference clip information 2801. On theother hand, if the stream path information 2807 indicates “sub-pathID=1”, the file 2D to which the selected elementary stream belongscorresponds to the 2D clip information file indicated by the referenceclip information of the SUB_PI included in the sub-path with a sub-pathID=1. The playback start time and playback end time specified by thisSUB_PI are both included in the interval from the playback start time2802 until the playback end time 2803 specified by the PI included inthe STN table 2805. The stream identification information 2808 indicatesthe PID for the elementary stream multiplexed in the file 2D specifiedby the stream path information 2807. The elementary stream indicated bythis PID can be selected from the playback start time 2802 until theplayback end time 2803. The stream attribute information 2810 indicatesattribute information for each elementary stream. For example, theattribute information for each of an audio stream, PG stream, and IGstream indicates a language type of the stream.

[Playback of 2D Video Images in Accordance with a 2D Playlist File]

FIG. 30 is a schematic diagram showing correspondence between the PTSsindicated by the 2D playlist file (00001.mpls) 221 and the sectionsplayed back from the file 2D (01000.m2ts) 241. As shown in FIG. 30, inthe main path 2701 in the 2D playlist file 221, the PI #1 specifies aPTS #1, which indicates a playback start time IN1, and a PTS #2, whichindicates a playback end time OUT1. The reference clip information forthe PI #1 indicates the 2D clip information file (01000.clpi) 231. Whenplaying back 2D video images in accordance with the 2D playlist file221, the playback device 102 first reads the PTS #1 and PTS #2 from thePI #1. Next, the playback device 102 refers to the entry map in the 2Dclip information file 231 to retrieve from the file 2D 241 the SPN #1and SPN #2 that correspond to the PTS #1 and PTS #2. The playback device102 then calculates the corresponding numbers of sectors from the SPN #1and SPN #2. Furthermore, the playback device 102 refers to these numbersof sectors and the file entry for the file 2D 241 to specify the LBN #1and LBN #2 at the beginning and end, respectively, of the sector groupP1 on which the 2D extent group EXT2D[0], . . . , EXT2D[n] to be playedback is recorded. Calculation of the numbers of sectors andspecification of the LBNs are as per the description on FIGS. 23B and23C. Finally, the playback device 102 indicates the range from LBN #1 toLBN #2 to the BD-ROM drive 121. The source packet group belonging to the2D extent group EXT2D[0], . . . , EXT2D[n] is thus read from the sectorgroup P1 in this range. Similarly, the pair PTS #3 and PTS #4 indicatedby the PI #2 are first converted into a pair of SPN #3 and SPN #4 byreferring to the entry map in the 2D clip information file 231. Then,referring to the file entry for the file 2D 241, the pair of SPN #3 andSPN #4 are converted into a pair of LBN #3 and LBN #4. Furthermore, asource packet group belonging to the 2D extent group is read from thesector group P2 in a range from the LBN #3 to the LBN #4. Conversion ofa pair of PTS #5 and PTS #6 indicated by the PI #3 to a pair of SPN #5and SPN #6, conversion of the pair of SPN #5 and SPN #6 to a pair of LBN#5 and LBN #6, and reading of a source packet group from the sectorgroup P3 in a range from the LBN #5 to the LBN #6 are similarlyperformed. The playback device 102 thus plays back 2D video images fromthe file 2D 241 in accordance with the main path 2701 in the 2D playlistfile 221.

The 2D playlist file 221 may include an entry mark 3001. The entry mark3001 indicates a time point in the main path 2701 at which playback isactually to start. For example, as shown in FIG. 30, a plurality ofentry marks 3001 can be set for the PI #1. The entry mark 3001 isparticularly used for searching for a playback start position duringrandom access. For example, when the 2D playlist file 221 specifies aplayback path for a movie title, the entry marks 3001 are assigned tothe top of each chapter. Consequently, the playback device 102 can playback the movie title by chapters.

<<3D Playlist File>>

FIG. 31 is a schematic diagram showing a data structure of a 3D playlistfile. The second playlist file (00002.mpls) 222 shown in FIG. 2 has thisdata structure, as does the third playlist file (00003.mpls) 223. Asshown in FIG. 31, the 3D playlist file 222 includes a main path 3101,sub-path 3102, and extension data 3103.

The main path 3101 specifies the playback path of the main TS shown inFIG. 3A. Accordingly, the main path 3101 is substantially the same asthe main path 2701 for the 2D playlist file 221 shown in FIG. 27. Inother words, the playback device 102 in 2D playback mode can play back2D video images from the file 2D 241 in accordance with the main path3101 in the 3D playlist file 222. The main path 3101 differs from themain path 2701 shown in FIG. 27 in that, when an STN is associated witha PID in a graphics stream, the STN table for each PI allocates anoffset sequence ID to the STN.

The sub-path 3102 specifies the playback path for the sub-TS shown inFIG. 3B, i.e. the playback path for the file DEP 242 or 243. The datastructure of the sub-path 3102 is the same as the data structure of thesub-paths 2702 and 2703 in the 2D playlist file 241 shown in FIG. 27.Accordingly, details of this similar data structure can be found in thedescription on FIG. 27, in particular details of the data structure ofthe SUB_PI.

The SUB_PI #N (N=1, 2, 3, . . . ) in the sub-path 3102 are in one-to-onecorrespondence with the PI #N in the main path 3101. Furthermore, theplayback start time and playback end time specified by each SUB_PI #N isthe same as the playback start time and playback end time specified bythe corresponding PI #N. The sub-path 3102 additionally includes asub-path type 3110. The “sub-path type” generally indicates whetherplayback processing should be synchronized between the main path and thesub-path. In the 3D playlist file 222, the sub-path type 3110 inparticular indicates the type of the 3D playback mode, i.e. the type ofthe dependent-view video stream to be played back in accordance with thesub-path 3102. In FIG. 31, the value of the sub-path type 3110 is “3DL/R”, thus indicating that the 3D playback mode is L/R mode, i.e. thatthe right-view video stream is to be played back. On the other hand, avalue of “3D depth” for the sub-path type 3110 indicates that the 3Dplayback mode is depth mode, i.e. that the depth map stream is to beplayed back. When the playback device 102 in 3D playback mode detectsthat the value of the sub-path type 3110 is “3D L/R” or “3D depth”, theplayback device 102 synchronizes playback processing that conforms tothe main path 3101 with playback processing that conforms to thesub-path 3102.

Extension data 3103 is interpreted only by the playback device 102 in 3Dplayback mode; the playback device 102 in 2D playback mode ignores theextension data 3103. In particular, the extension data 3103 includes anextension stream selection table 3130. The “extension stream selectiontable (STN_table_SS)” (hereinafter abbreviated as “STN table SS”) is anarray of stream registration information to be added to the STN tablesindicated by each PI in the main path 3101 during 3D playback mode. Thisstream registration information indicates elementary streams that can beselected for playback from the sub TS.

[STN Table]

FIG. 32 is a schematic diagram showing an STN table 3205 included in amain path 3101 of the 3D playlist file 222. As shown in FIG. 32, thestream identification information 3208 allocated to STN 3206=5, 6, . . ., 11 indicates PIDs for a PG stream, or IG stream. In this case, thestream attribute information 3210 allocated to the STN 3206=5, 6, . . ., 11 further includes a reference offset ID 3201 and offset adjustmentvalue 3202.

In the file DEP 242, as shown in FIG. 11, offset metadata 1110 is placedin VAU #1 of each video sequence. The reference offset ID(stream_ref_offset_id) 3201 is the same as one of the offset sequenceIDs 1111 included in the offset metadata 1110. In other words, thereference offset ID 3201 defines the offset sequence that should beassociated with each of the STNs 3206=5, 6, . . . , 11 from among theplurality of offset sequences included in the offset metadata 1110.

The offset adjustment value (stream_offset_adjustment) 3202 indicatesthe value that should be added to each offset value included in theoffset sequence defined by the reference offset ID 3201. The offsetadjustment value 3202 is, for example, added by the playback device 102to each offset value when the size of the screen of the display device103 differs from the size that was assumed during creation of the 3Dvideo content. In this way, the binocular parallax between 2D graphicsimages for a left view and a right view can be maintained in anappropriate range.

[STN Table SS]

FIG. 33 is a schematic diagram showing a data structure of the STN tableSS 3130. As shown in FIG. 33, an STN table SS 3130 includes streamregistration information sequences 3301, 3302, 3303, . . . . The streamregistration information sequences 3301, 3302, 3303, . . . individuallycorrespond to the PI #1, PI #2, PI #3, . . . in the main path 3101 andare used by the playback device 102 in 3D playback mode in combinationwith the stream registration information sequences included in the STNtables in the corresponding PIs. The stream registration informationsequence 3301 corresponding to each PI includes an offset during pop-up(Fixed_offset_during_Popup) 3311 and stream registration informationsequence 3312 for the dependent-view video streams.

The offset during pop-up 3311 indicates whether a pop-up menu is playedback from the IG stream. The playback device 102 in 3D playback modechanges the presentation mode of the video plane and the graphics planein accordance with the value of the offset 3311. There are two types ofpresentation modes for the video plane: base-view (B)-dependent-view (D)presentation mode and B-B presentation mode. There are two types ofpresentation modes for the graphics plane: 1 plane+offset mode and 1plane+zero offset mode. For example, when the value of the offset duringpop-up 3311 is “0”, a pop-up menu is not played back from the IG stream.At this point, B-D presentation mode is selected as the video planepresentation mode, and 1 plane+offset mode is selected as thepresentation mode for the graphics plane. On the other hand, when thevalue of the offset during pop-up 3311 is “1”, a pop-up menu is playedback from the IG stream. At this point, B-B presentation mode isselected as the video plane presentation mode, and 1 plane+zero offsetmode is selected as the presentation mode for the graphics plane.

In “B-D presentation mode”, the playback device 102 alternately outputsthe left-view and right-view video planes. Accordingly, since left-viewand right-view frames are alternately displayed on the screen of thedisplay device 103, the viewer perceives these frames as 3D videoimages. In “B-B presentation mode”, the playback device 102 outputsplane data decoded only from the base-view video stream twice for aframe while maintaining the operation mode in 3D playback mode (inparticular, maintaining the frame rate at the value for 3D playback,e.g. 48 frames/second). Accordingly, only either the left-view orright-view video plane is displayed on the screen of the display device103, and thus the viewer perceives these video planes simply as 2D videoimages.

In “1 plane+offset mode”, the playback device 102 generates, via offsetcontrol, a pair of left-view and right-view graphics planes from thegraphics stream in the main TS and alternately outputs these graphicsplanes. Accordingly, left-view and right-view graphics planes arealternately displayed on the screen of the display device 103, and thusthe viewer perceives these frames as 3D graphics images. In “1plane+zero offset mode”, the playback device 102 temporarily stopsoffset control and outputs a graphics plane decoded from the graphicsstream in the main TS twice for a frame while maintaining the operationmode in 3D playback mode. Accordingly, only either the left-view orright-view graphics planes are displayed on the screen of the displaydevice 103, and thus the viewer perceives these planes simply as 2Dgraphics images.

The playback device 102 in 3D playback mode refers to the offset duringpop-up 3311 for each PI and selects B-B presentation mode and 1plane+zero offset mode when a pop-up menu is played back from an IGstream. While a pop-up menu is displayed, other 3D video images are thustemporarily changed to 2D video images. This improves the visibility andusability of the pop-up menu.

The stream registration information sequence 3312 for the dependent-viewvideo stream includes stream registration information indicating thedependent-view video streams that can be selected for playback from thesub-TS. This stream registration information sequence 3312 is used incombination with one of the stream registration information sequencesincluded in the STN table in the corresponding PI, which indicates thebase-view video stream. When reading a piece of stream registrationinformation from an STN table, the playback device 102 in 3D playbackmode automatically also reads the stream registration informationsequence, located in the STN table SS, that has been combined with thepiece of stream registration information. When simply switching from 2Dplayback mode to 3D playback mode, the playback device 102 can thusmaintain already recognized STNs and stream attributes such as language.

As shown in FIG. 33, the stream registration information sequence 3312in the dependent-view video stream generally includes a plurality ofpieces of stream registration information (SS_dependent_view_block)3320. These are the same in number as the pieces of stream registrationinformation in the corresponding PI that indicate the base-view videostream. Each piece of stream registration information 3320 includes anSTN 3321, stream entry 3322, and stream attribute information 3323. TheSTN 3321 is a serial number assigned individually to pieces of streamregistration information 3320 and is the same as the STN of the piece ofstream registration information, located in the corresponding PI, withwhich the piece of stream registration information 3320 is combined. Thestream entry 3322 includes sub-path ID reference information(ref_to_Subpath_id) 3331, stream file reference information(ref_to_subClip_entry_id) 3332, and a PID (ref_to_stream_PID_subclip)3333. The sub-path ID reference information 3331 indicates the sub-pathID of the sub-path that specifies the playback path of thedependent-view video stream. The stream file reference information 3332is information to identify the file DEP storing this dependent-viewvideo stream. The PID 3333 is the PID for this dependent-view videostream. The stream attribute information 3323 includes attributes forthis dependent-view video stream, such as frame rate, resolution, andvideo format. In particular, these attributes are the same as attributesfor the base-view video stream shown by the piece of stream registrationinformation, included in the corresponding PI, with which each piece ofstream registration information is combined.

[Playback of 3D Video Images in Accordance With a 3D Playlist File]

FIG. 34 is a schematic diagram showing correspondence between PTSsindicated by the 3D playlist file (00002.mpls) 222 and sections playedback from the file SS (01000.ssif) 244A. As shown in FIG. 34, in themain path 3401 in the 3D playlist file 222, the PI #1 specifies a PTS#1, which indicates a playback start time IN1, and a PTS #2, whichindicates a playback end time OUT1. The reference clip information forthe PI #1 indicates the 2D clip information file (01000.clpi) 231. Inthe sub-path 3402, which indicates that the sub-path type is “3D L/R”,the SUB_PI #1 specifies the same PTS #1 and PTS #2 as the PI #1. Thereference clip information for the SUB_PI #1 indicates thedependent-view clip information file (02000.clpi) 232.

When playing back 3D video images in accordance with the 3D playlistfile 222, the playback device 102 first reads PTS #1 and PTS #2 from thePI #1 and SUB_PI #1. Next, the playback device 102 refers to the entrymap in the 2D clip information file 231 to retrieve from the file 2D 241the SPN #1 and SPN #2 that correspond to the PTS #1 and PTS #2. Inparallel, the playback device 102 refers to the entry map in thedependent-view clip information file 232 to retrieve from the first fileDEP 242 the SPN #11 and SPN #12 that correspond to the PTS #1 and PTS#2. As described with reference to FIG. 24E, the playback device 102then uses the extent start points 2242 and 2420 in the clip informationfiles 231 and 232 to calculate, from SPN #1 and SPN #11, the number ofsource packets SPN #21 from the top of the file SS 244A to the playbackstart position. Similarly, the playback device 102 calculates, from SPN#2 and SPN #12, the number of source packets SPN #22 from the top of thefile SS 244A to the playback start position. The playback device 102further calculates the numbers of sectors corresponding to the SPN #21and SPN #22. Next, the playback device 102 refers to these numbers ofsectors and the file entry of the file SS 244A to specify the LBN #1 andLBN #2 at the beginning and end, respectively, of the sector group P11on which the extent SS group EXTSS[0], . . . , EXTSS[n] to be playedback is recorded. Calculation of the numbers of sectors andspecification of the LBNs are as per the description on FIG. 24E.Finally, the playback device 102 indicates the range from LBN #1 to LBN#2 to the BD-ROM drive 121. The source packet group belonging to theextent SS group EXTSS[0], . . . , EXTSS[n] is thus read from the sectorgroup P11 in this range. Similarly, the pair PTS #3 and PTS #4 indicatedby the PI #2 and SUB_PI #2 are first converted into a pair of SPN #3 andSPN #4 and a pair of SPN #13 and SPN #14 by referring to the entry mapsin the clip information files 231 and 232. Then, the number of sourcepackets SPN #23 from the top of the file SS 244A to the playback startposition is calculated from SPN #3 and SPN #13, and the number of sourcepackets SPN #24 from the top of the file SS 244A to the playback endposition is calculated from SPN #4 and SPN #14. Next, referring to thefile entry for the file SS 244A, the pair of SPN #23 and SPN #24 areconverted into a pair of LBN #3 and LBN #4. Furthermore, a source packetgroup belonging to the extent SS group is read from the sector group P12in a range from the LBN #3 to the LBN #4.

In parallel with the above-described read processing, as described withreference to FIG. 24E, the playback device 102 refers to the extentstart points 2242 and 2420 in the clip information files 231 and 232 toextract base-view extents from each extent SS and decode the base-viewextents in parallel with the remaining dependent-view extents. Theplayback device 102 can thus play back 3D video images from the file SS244A in accordance with the 3D playlist file 222.

<<Index File>>

FIG. 35 is a schematic diagram showing a data structure of an index file(index.bdmv) 211 shown in FIG. 2. As shown in FIG. 35, the index file211 includes an index table 3510, 3D existence flag 3520, and 2D/3Dpreference flag 3530.

The index table 3510 stores the items “first play” 3501, “top menu”3502, and “title k” 3503 (k=1, 2, . . . , n; the letter n represents aninteger greater than or equal to 1). Each item is associated with eithera movie object MVO-2D, MVO-3D, . . . , or a BD-J object BDJO-2D,BDJO-3D, . . . . Each time a title or a menu is called in response to auser operation or an application program, a control unit in the playbackdevice 102 refers to a corresponding item in the index table 3510.Furthermore, the control unit calls an object associated with the itemfrom the BD-ROM disc 101 and accordingly executes a variety ofprocesses. Specifically, the item “first play” 3501 specifies an objectto be called when the disc 101 is loaded into the BD-ROM drive 121. Theitem “top menu” 3502 specifies an object for displaying a menu on thedisplay device 103 when a command “go back to menu” is input, forexample, by user operation. In the items “title k” 3503, the titles thatconstitute the content on the disc 101 are individually allocated. Forexample, when a title for playback is specified by user operation, inthe item “title k” in which the title is allocated, the object forplaying back video images from the AV stream file corresponding to thetitle is specified.

In the example shown in FIG. 35, the items “title 1” and “title 2” areallocated to titles of 2D video images. The movie object associated withthe item “title 1”, MVO-2D, includes a group of commands related toplayback processes for 2D video images using the 2D playlist file(00001.mpls) 221. When the playback device 102 refers to the item “title1”, then in accordance with the movie object MVO-2D, the 2D playlistfile 221 is read from the disc 101, and playback processes for 2D videoimages are executed in accordance with the playback path specifiedtherein. The BD-J object associated with the item “title 2”, BDJO-2D,includes an application management table related to playback processesfor 2D video images using the 2D playlist file 221. When the playbackdevice 102 refers to the item “title 2”, then in accordance with theapplication management table in the BD-J object BDJO-2D, a Javaapplication program is called from the JAR file 261 and executed. Inthis way, the 2D playlist file 221 is read from the disc 101, andplayback processes for 2D video images are executed in accordance withthe playback path specified therein.

Furthermore, in the example shown in FIG. 35, the items “title 3” and“title 4” are allocated to titles of 3D video images. The movie objectassociated with the item “title 3”, MVO-3D, includes, in addition to agroup of commands related to playback processes for 2D video imagesusing the 2D playlist file 221, a group of commands related to playbackprocesses for 3D video images using either 3D playlist file (00002.mpls)222 or (00003.mpls) 223. In the BD-J object associated with the item“title 4”, BDJO-3D, the application management table specifies, inaddition to a Java application program related to playback processes for2D video images using the 2D playlist file 221, a Java applicationprogram related to playback processes for 3D video images using either3D playlist file 222 or 223.

The 3D existence flag 3520 shows whether or not 3D video content isrecorded on the BD-ROM disc 101. When the BD-ROM disc 101 is insertedinto the BD-ROM drive 121, the playback device 102 first checks the 3Dexistence flag 3520. When the 3D existence flag 3520 is off, theplayback device 102 does not need to select 3D playback mode.Accordingly, the playback device 102 can rapidly proceed in 2D playbackmode without performing HDMI authentication on the display device 103.“HDMI authentication” refers to processing by which the playback device102 exchanges CEC messages with the display device 103 via the HDMIcable 122 to check with the display device 103 as to whether it supportsplayback of 3D video images. By skipping HDMI authentication, the timebetween insertion of the BD-ROM disc 101 and the start of playback of 2Dvideo images is shortened.

The 2D/3D preference flag 3530 indicates whether playback of 3D videoimages should be prioritized when both the playback device and thedisplay device support playback of both 2D video images and 3D videoimages. The 2D/3D preference flag 3530 is set by the content provider.When the 3D existence flag 3520 in the BD-ROM disc 101 is on, theplayback device 102 then additionally checks the 2D/3D preference flag3530. When the 2D/3D preference flag 3530 is on, the playback device 102does not make the user select the playback mode, but rather performsHDMI authentication. Based on the results thereof, the playback device102 operates in either 2D playback mode or 3D playback mode. That is,the playback device 102 does not display a playback mode selectionscreen. Accordingly, if the results of HDMI authentication indicate thatthe display device 103 supports playback of 3D video images, theplayback device 102 operates in 3D playback mode. This makes it possibleto avoid delays in starting up caused by processing to switch from 2Dplayback mode to 3D playback mode, such as switching frame rates, etc.

[Selection of Playlist File when Selecting a 3D Video Title]

In the example shown in FIG. 35, when the playback device 102 refers toitem “title 3” in the index table 3510, the following determinationprocesses are performed in accordance with the movie object MVO-3D: (1)Is the 3D existence flag 3520 on or off?(2) Does the playback device 102itself support playback of 3D video images? (3) Is the 2D/3D preferenceflag 3530 on or off? (4) Has the user selected 3D playback mode? (5)Does the display device 103 support playback of 3D video images? and (6)Is the 3D playback mode of the playback device 102 in L/R mode or depthmode? Next, in accordance with the results of these determinations, theplayback device 102 selects one of the playlist files 221-223 forplayback. On the other hand, when the playback device 102 refers to item“title 4”, a Java application program is called from the JAR file 261,in accordance with the application management table in the BD-J objectBDJO-3D, and executed. The above-described determination processes(1)-(6) are thus performed, and a playlist file is then selected inaccordance with the results of determination.

FIG. 36 is a flowchart of selection processing for a playlist file to beplayed back using the above determination processes (1)-(6). For thisselection processing, it is assumed that the playback device 102includes a first flag and a second flag. The first flag indicateswhether the playback device 102 supports playback of 3D video images.For example, a value of “0” for the first flag indicates that theplayback device 102 only supports playback of 2D video images, whereas“1” indicates support of 3D video images as well. The second flagindicates whether the 3D playback mode is L/R mode or depth mode. Forexample, a value of “0” for the second flag indicates that the 3Dplayback mode is L/R mode, whereas “1” indicates depth mode.Furthermore, the respective values of the 3D existence flag 3520 and2D/3D preference flag 3530 are set to “1” when these flags are on, andto “0” when these flags are off.

In step S3601, the playback device 102 checks the value of the 3Dexistence flag 3520. If the value is “1”, processing proceeds to stepS3602. If the value is “0”, processing proceeds to step S3607.

In step S3602, the playback device 102 checks the value of the firstflag. If the value is “1”, processing proceeds to step S3603. If thevalue is “0”, processing proceeds to step S3607.

In step S3603, the playback device 102 checks the value of the 2D/3Dpreference flag 3530. If the value is “0”, processing proceeds to stepS3604. If the value is “1”, processing proceeds to step S3605.

In step S3604, the playback device 102 displays a menu on the displaydevice 103 for the user to select either 2D playback mode or 3D playbackmode. If the user selects 3D playback mode via operation of a remotecontrol 105 or the like, processing proceeds to step S3605, whereas ifthe user selects 2D playback mode, processing proceeds to step S3607.

In step S3605, the playback device 102 perform HDMI authentication tocheck whether the display device 103 supports playback of 3D videoimages. Specifically, the playback device 102 exchanges CEC messageswith the display device 103 via the HDMI cable 122 to check with thedisplay device 103 as to whether it supports playback of 3D videoimages. If the display device 103 does support playback of 3D videoimages, processing proceeds to step S3606. If the display device 103does not support playback of 3D video images, processing proceeds tostep S3607.

In step S3606, the playback device 102 checks the value of the secondflag. If the value is “0”, processing proceeds to step S3608. If thevalue is “1”, processing proceeds to step S3609.

In step S3607, the playback device 102 selects for playback the 2Dplaylist file 221. Note that, at this time, the playback device 102 maycause the display device 103 to display the reason why playback of 3Dvideo images was not selected. Processing then terminates.

In step S3608, the playback device 102 selects for playback the 3Dplaylist file 222 used in L/R mode. Processing then terminates.

In step S3609, the playback device 102 selects for playback the 3Dplaylist file 222 used in depth mode. Processing then terminates.

<Structure of 2D Playback Device>

When playing back 2D video content from the BD-ROM disc 101 in 2Dplayback mode, the playback device 102 operates as a 2D playback device.FIG. 37 is a functional block diagram of a 2D playback device 3700. Asshown in FIG. 37, the 2D playback device 3700 includes a BD-ROM drive3701, playback unit 3702, and control unit 3703. The playback unit 3702includes a read buffer 3721, system target decoder 3725, and plane adder3726. The control unit 3703 includes a dynamic scenario memory 3731,static scenario memory 3732, user event processing unit 3733, programexecution unit 3734, playback control unit 3735, and player variablestorage unit 3736. The playback unit 3702 and the control unit 3703 areeach implemented on a different integrated circuit, but mayalternatively be implemented on a single integrated circuit.

When the BD-ROM disc 101 is loaded into the BD-ROM drive 3701, theBD-ROM drive 3701 radiates laser light to the disc 101 and detectschange in the reflected light. Furthermore, using the change in theamount of reflected light, the BD-ROM drive 3701 reads data recorded onthe disc 101. Specifically, the BD-ROM drive 3701 has an optical pickup,i.e. an optical head. The optical head has a semiconductor laser,collimate lens, beam splitter, objective lens, collecting lens, andoptical detector. A beam of light radiated from the semiconductor lasersequentially passes through the collimate lens, beam splitter, andobjective lens to be collected on a recording layer of the disc 101. Thecollected beam is reflected and diffracted by the recording layer. Thereflected and diffracted light passes through the objective lens, thebeam splitter, and the collecting lens, and is collected onto theoptical detector. The optical detector generates a playback signal at alevel in accordance with the amount of collected light. Furthermore,data is decoded from the playback signal.

The BD-ROM drive 3701 reads data from the BD-ROM disc 101 based on arequest from the playback control unit 3735. Out of the read data, theextents in the file 2D, i.e. the 2D extents, are transferred to the readbuffer 3721; dynamic scenario information is transferred to the dynamicscenario memory 3731; and static scenario information is transferred tothe static scenario memory 3732. “Dynamic scenario information” includesan index file, movie object file, and BD-J object file. “Static scenarioinformation” includes a 2D playlist file and a 2D clip information file.

The read buffer 3721, dynamic scenario memory 3731, and static scenariomemory 3732 are each a buffer memory. Memory elements in the playbackunit 3702 are used as the read buffer 3721. Memory elements in thecontrol unit 3703 are used as the dynamic scenario memory 3731 and thestatic scenario memory 3732. Alternatively, different areas in a singlememory element may be used as part or all of these buffer memories 3721,3731, and 3732.

The system target decoder 3725 reads 2D extents from the read buffer3721 in units of source packets and demultiplexes the 2D extents. Thesystem target decoder 3725 then decodes each of the elementary streamsobtained by the demultiplexing. At this point, information necessary fordecoding each elementary stream, such as the type of codec andattributes of the stream, is transferred from the playback control unit3735 to the system target decoder 3725. The system target decoder 3725outputs a primary video stream, secondary video stream, IG stream, andPG stream after decoding respectively as primary video plane data,secondary video plane data, IG plane data, and PG plane data, in unitsof VAUs. On the other hand, the system target decoder 3725 mixes thedecoded primary audio stream and secondary audio stream and transmitsthe resultant data to an audio output device, such as an internalspeaker 103A of the display device 103. In addition, the system targetdecoder 3725 receives graphics data from the program execution unit3734. The graphics data is used for rendering graphic elements for aGUI, such as a menu, on the screen and is in a raster data format suchas JPEG and PNG. The system target decoder 3725 processes the graphicsdata and outputs the processed data as image plane data. Details on thesystem target decoder 3725 are provided below.

The plane adder 3726 receives primary video plane data, secondary videoplane data, IG plane data, PG plane data, and image plane data from thesystem target decoder 3725 and superimposes these pieces of plane datato generate one combined video frame or field. The combined video datais transferred to the display device 103 for display on the screen.

The user event processing unit 3733 detects a user operation via theremote control 105 or the front panel of the playback device 102. Basedon the user operation, the user event processing unit 3733 requests theprogram execution unit 3734 or the playback control unit 3735 to performprocessing. For example, when a user instructs to display a pop-up menuby pushing a button on the remote control 105, the user event processingunit 3733 detects the push and identifies the button. The user eventprocessing unit 3733 further requests the program execution unit 3734 toexecute a command corresponding to the button, i.e. a command to displaythe pop-up menu. On the other hand, when a user pushes a fast-forward ora rewind button on the remote control 105, for example, the user eventprocessing unit 3733 detects the push and identifies the button. Theuser event processing unit 3733 then requests the playback control unit3735 to fast-forward or rewind the playlist currently being played back.

The program execution unit 3734 is a processor that reads programs frommovie object files and BD-J object files stored in the dynamic scenariomemory 3731 and executes these programs. Furthermore, the programexecution unit 3734 performs the following operations in accordance withthe programs: (1) The program execution unit 3734 orders the playbackcontrol unit 3735 to perform playlist playback processing; (2) Theprogram execution unit 3734 generates graphics data for a menu or gameas PNG or JPEG raster data and transfers the generated data to thesystem target decoder 3725 to be combined with other video data. Viaprogram design, specific details on these processes can be designedrelatively flexibly. In other words, during the authoring process of theBD-ROM disc 101, the nature of these processes is determined whileprogramming the movie object files and BD-J object files.

The playback control unit 3735 controls transfer of different types ofdata, such as 2D extents, an index file, etc. from the BD-ROM disc 101to the read buffer 3721, dynamic scenario memory 3731, and staticscenario memory 3732. A file system managing the directory filestructure shown in FIG. 2 is used for this control. That is, theplayback control unit 3735 causes the BD-ROM drive 3701 to transfer thefiles to each of the buffer memories 3721, 3731, and 3732 using a systemcall for opening files. The “file opening” is composed of a sequence ofthe following processes. First, a file name to be detected is providedto the file system by a system call, and an attempt is made to detectthe file name from the directory/file structure. When the detection issuccessful, the file entry for the target file to be transferred isfirst transferred to memory in the playback control unit 3735, and aFile Control Block (FCB) is generated in the memory. Subsequently, afile handle for the target file is returned from the file system to theplayback control unit 3735. Afterwards, the playback control unit 3735can cause the BD-ROM drive 3701 to transfer the target file from theBD-ROM disc 101 to each of the buffer memories 3721, 3731, and 3732 byshowing the file handle to the BD-ROM drive 3701.

The playback control unit 3735 decodes the file 2D to output video dataand audio data by controlling the BD-ROM drive 3701 and the systemtarget decoder 3725. Specifically, the playback control unit 3735 firstreads a 2D playlist file from the static scenario memory 3732, inresponse to an instruction from the program execution unit 3734 or arequest from the user event processing unit 3733, and interprets thecontent of the file. In accordance with the interpreted content,particularly with the playback path, the playback control unit 3735 thenspecifies a file 2D to be played back and instructs the BD-ROM drive3701 and the system target decoder 3725 to read and decode this file.Such playback processing based on a playlist file is referred to as“playlist playback processing”.

In addition, the playback control unit 3735 sets various types of playervariables in the player variable storage unit 3736 using the staticscenario information. With reference to the player variables, theplayback control unit 3735 further specifies to the system targetdecoder 3725 elementary streams to be decoded and provides theinformation necessary for decoding the elementary streams.

The player variable storage unit 3736 is composed of a group ofregisters for storing player variables. Types of player variablesinclude system parameters (SPRM) and general parameters (GPRM). An SPRMindicates the status of the playback device 102. FIG. 38 is a list ofSPRMs. As shown in FIG. 38, each SPRM is assigned a serial number 3801,and each serial number 3801 is associated with a unique variable value3802. There are provided, for example, 64 SPRMs. The contents of SPRMsare shown below. Here, the numbers in parentheses indicate the serialnumbers 3801.

SPRM(0): Language code

SPRM(1): Primary audio stream number

SPRM(2): Subtitle stream number

SPRM(3): Angle number

SPRM(4): Title number

SPRM(5): Chapter number

SPRM(6): Program number

SPRM(7): Cell number

SPRM(8): Key name

SPRM(9): Navigation timer

SPRM(10): Current playback time

SPRM(11): Player audio mixing mode for karaoke

SPRM(12): Country code for parental management

SPRM(13): Parental level

SPRM(14): Player configuration for video

SPRM(15): Player configuration for audio

SPRM(16): Language code for audio stream

SPRM(17): Language code extension for audio stream

SPRM(18): Language code for subtitle stream

SPRM(19): Language code extension for subtitle stream

SPRM(20): Player region code

SPRM(21): Secondary video stream number

SPRM(22): Secondary audio stream number

SPRM(23): Player status

SPRM(24)—SPRM(63): Reserved

The SPRM(10) indicates the PTS of the picture currently being decodedand is updated every time a picture is decoded and written into theprimary video plane memory. Accordingly, the current playback point canbe known by referring to the SPRM(10).

The parental level in SPRM(13) indicates a predetermined restricted ageand is used for parental control of viewing of titles recorded on theBD-ROM disc 101. A user of the playback device 102 sets the value of theSPRM(13) via, for example, an OSD of the playback device 102. “Parentalcontrol” refers to restricting viewing of a title in accordance with theviewer's age. The following is an example of how the playback device 102performs parental control. The playback device 102 first reads the agefor which viewing of a title is permitted from the BD-ROM disc 101, andthen compares this age with the value of the SPRM(13). If this age isequal to or less than the value of the SPRM(13), the playback device 102continues with playback of the title. If this age is greater than thevalue of the SPRM(13), the playback device 102 stops playback of thetitle.

The language code for audio stream in SPRM(16) and the language code forsubtitle stream in SPRM(18) show default language codes of the playbackdevice 102. These codes may be changed by a user with use of the OSD orthe like of the playback device 102, or the codes may be changed by anapplication program via the program execution unit 3734. For example, ifthe SPRM(16) shows “English”, then during playback processing of aplaylist, the playback control unit 3735 first searches the STN table inthe PI showing the current playback section, i.e. the current PI, for astream entry having the language code for “English”. The playbackcontrol unit 3735 then extracts the PID from the stream identificationinformation of the stream entry and transmits the extracted PID to thesystem target decoder 3725. As a result, an audio stream having the PIDis selected and decoded by the system target decoder 3725. Theseprocesses can be executed by the playback control unit 3735 with use ofthe movie object file or the BD-J object file.

During playback processing, the playback control unit 3735 updates theplayer variables in accordance with the status of playback. The playbackcontrol unit 3735 updates the SPRM(1), SPRM(2), SPRM(21), and SPRM(22)in particular. These SPRM respectively show, in the stated order, theSTN of the audio stream, subtitle stream, secondary video stream, andsecondary audio stream that are currently being processed. For example,suppose that the SPRM(1) has been changed by the program execution unit3734. In this case, the playback control unit 3735 first refers to theSTN shown by the new SPRM(1) and retrieves the stream entry thatincludes this STN from the STN table in the current PI. The playbackcontrol unit 3735 then extracts the PID from the stream identificationinformation of the stream entry and transmits the extracted PID to thesystem target decoder 3725. As a result, an audio stream having the PIDis selected and decoded by the system target decoder 3725. This is howthe audio stream to be played back is switched. The subtitle stream andthe secondary video stream to be played back can be similarly switched.

<<2D Playlist Playback Processing>>

FIG. 39 is a flowchart of 2D playlist playback processing by a playbackcontrol unit 3735. 2D playlist playback processing is performedaccording to a 2D playlist file and is started by the playback controlunit 3735 reading a 2D playlist file from the static scenario memory3732.

In step S3901, the playback control unit 3735 first reads a single PIfrom a main path in the 2D playlist file and then sets the PI as thecurrent PI. Next, from the STN table of the current PI, the playbackcontrol unit 3735 selects PIDs of elementary streams to be played backand specifies attribute information necessary for decoding theelementary streams. The selected PIDs and attribute information areindicated to the system target decoder 3725. The playback control unit3735 further specifies a SUB_PI associated with the current PI from thesub-paths in the 2D playlist file. Thereafter, processing proceeds tostep S3902.

In step S3902, the playback control unit 3735 reads reference clipinformation, a PTS #1 indicating a playback start time IN1, and a PTS #2indicating a playback end time OUT1 from the current PI. From thisreference clip information, a 2D clip information file corresponding tothe file 2D to be played back is specified. Furthermore, when a SUB_PIexists that is associated with the current PI, similar information isalso read from the SUB_PI. Thereafter, processing proceeds to stepS3903.

In step S3903, with reference to the entry map of the 2D clipinformation file, the playback control unit 3735 retrieves the SPN #1and the SPN #2 in the file 2D corresponding to the PTS #1 and the PTS#2. The pair of PTSs indicated by the SUB_PI are also converted to apair of SPNs. Thereafter, processing proceeds to step S3904.

In step S3904, from the SPN #1 and the SPN #2, the playback control unit3735 calculates a number of sectors corresponding to each of the SPN #1and the SPN #2. Specifically, the playback control unit 3735 firstobtains the product of each of the SPN #1 and the SPN #2 multiplied bythe data amount per source packet, i.e. 192 bytes. Next, the playbackcontrol unit 3735 obtains a quotient by dividing each product by thedata amount per sector, i.e. 2048 bytes: N1=SPN #1×192/2048, N2=SPN#2×192/2048. The quotients N1 and N2 are the same as the total number ofsectors, in the main TS, recorded in portions previous to the sourcepackets to which SPN #1 and SPN #2 are allocated, respectively. The pairof SPNs converted from the pair of PTSs indicated by the SUB_PI issimilarly converted to a pair of numbers of sectors. Thereafter,processing proceeds to step S3905.

In step S3905, the playback control unit 3735 specifies, from thenumbers of sectors N1 and N2 obtained in step S3904, LBNs of the top andend of the 2D extent group to be played back. Specifically, withreference to the file entry of the file 2D to be played back, theplayback control unit 3735 counts from the top of the sector group inwhich the 2D extent group is recorded so that the LBN of the (N1+1)^(th)sector=LBN #1, and the LBN of the (N2+1)^(th) sector=LBN #2. Theplayback control unit 3735 further specifies a range from the LBN#1 tothe LBN#2 to the BD-ROM drive 121. The pair of numbers of sectorsconverted from the pair of PTSs indicated by the SUB_PI is similarlyconverted to a pair of LBNs and specified to the BD-ROM drive 121. As aresult, from the sector group in the specified range, a source packetgroup belonging to a 2D extent group is read in aligned units.Thereafter, processing proceeds to step S3906.

In step S3906, the playback control unit 3735 checks whether anunprocessed PI remains in the main path. When an unprocessed PI remains,processing is repeated from step S3901. When no unprocessed PI remains,processing ends.

<<System Target Decoder>>

FIG. 40 is a functional block diagram of the system target decoder 3725.As shown in FIG. 40, the system target decoder 3725 includes a sourcedepacketizer 4010, ATC counter 4020, first 27 MHz clock 4030, PID filter4040, STC counter (STC1) 4050, second 27 MHz clock 4060, primary videodecoder 4070, secondary video decoder 4071, PG decoder 4072, IG decoder4073, primary audio decoder 4074, secondary audio decoder 4075, textsubtitle decoder 4076, image processor 4080, primary video plane memory4090, secondary video plane memory 4091, PG plane memory 4092, IG planememory 4093, image plane memory 4094, and audio mixer 4095.

The source depacketizer 4010 reads source packets from the read buffer3721, extracts the TS packets from the read source packets, andtransfers the TS packets to the PID filter 4040. Furthermore, the sourcedepacketizer 4010 synchronizes the time of the transfer with the timeshown by the ATS of each source packet. Specifically, the sourcedepacketizer 4010 first monitors the value of the ATC generated by theATC counter 4020. In this case, the value of the ATC depends on the ATCcounter 4020 and is incremented in accordance with a pulse of a clocksignal from the first 27 MHz clock 4030. Subsequently, at the instantthe value of the ATC matches the ATS of a source packet, the sourcedepacketizer 4010 transfers the TS packets extracted from the sourcepacket to the PID filter 4040. By adjusting the time of transfer in thisway, the mean transfer rate of TS packets from the source depacketizer4010 to the PID filter 4040 does not surpass the value R_(TS) specifiedby the system rate 2211 in the 2D clip information file 231 shown inFIG. 22.

The PID filter 4040 first monitors a PID that includes each TS packetoutputted by the source depacketizer 4010. When the PID matches a PIDpre-specified by the playback control unit 3735, the PID filter 4040selects the TS packet and transfers it to the decoder 4070-4075appropriate for decoding of the elementary stream indicated by the PID(the text subtitle decoder 4076, however, is excluded). For example, ifa PID is 0x1011, the TS packets are transferred to the primary videodecoder 4070. TS packets with PIDs ranging from 0x1B00-0x1B1F,0x1100-0x111F, 0x1A00-0x1A1F, 0x1200-0x121F, and 0x1400-0x141F aretransferred to the secondary video decoder 4071, primary audio decoder4074, secondary audio decoder 4075, PG decoder 4072, and IG decoder4073, respectively.

The PID filter 4040 further detects a PCR from TS packets using the PIDsof the TS packets. At each detection, the PID filter 4040 sets the valueof the STC counter 4050 to a predetermined value. Then, the value of theSTC counter 4050 is incremented in accordance with a pulse of the clocksignal of the second 27 MHz clock 4060. In addition, the value to whichthe STC counter 4050 is set is indicated to the PID filter 4040 from theplayback control unit 3735 in advance. The decoders 4070-4076 each usethe value of the STC counter 4050 as the STC. Specifically, the decoders4070-4076 first reconstruct the TS packets received from the PID filter4040 into PES packets. Next, the decoders 4070-4076 adjust the timing ofthe decoding of data included in the PES payloads in accordance with thetimes indicated by the PTSs or the DTSs included in the PES headers.

The primary video decoder 4070, as shown in FIG. 40, includes atransport stream buffer (TB) 4001, multiplexing buffer (MB) 4002,elementary stream buffer (EB) 4003, compressed video decoder (DEC) 4004,and decoded picture buffer (DPB) 4005.

The TB 4001, MB 4002, and EB 4003 are each a buffer memory and use anarea of a memory element internally provided in the primary videodecoder 4070. Alternatively, some or all of the buffer memories may beseparated in discrete memory elements. The TB 4001 stores the TS packetsreceived from the PID filter 4040 as they are. The MB 4002 stores PESpackets reconstructed from the TS packets stored in the TB 4001. Notethat when the TS packets are transferred from the TB 4001 to the MB4002, the TS header is removed from each TS packet. The EB 4003 extractsencoded VAUs from the PES packets and stores the VAUs therein. A VAUincludes a compressed picture, i.e., an I picture, B picture, or Ppicture. Note that when data is transferred from the MB 4002 to the EB4003, the PES header is removed from each PES packet.

The DEC 4004 is a hardware decoder specifically for decoding ofcompressed pictures and is composed of an LSI that includes, inparticular, a function to accelerate the decoding. The DEC 4004 decodesa picture from each VAU in the EB 4003 at the time shown by the DTSincluded in the original PES packet. The DEC 4004 may also refer to thedecoding switch information 1250 shown in FIG. 12 to decode picturesfrom VAUs sequentially, regardless of the DTSs. During decoding, the DEC4004 first analyzes the VAU header to specify the compressed picture,compression encoding method, and stream attribute stored in the VAU,selecting a decoding method in accordance with this information.Compression encoding methods include, for example, MPEG-2, MPEG-4 AVC,and VC1. Furthermore, the DEC 4004 transmits the decoded, uncompressedpicture to the DPB 4005.

Like the TB 4001, MB 4002, and EB 4003, the DPB 4005 is a buffer memorythat uses an area of a built-in memory element in the primary videodecoder 4070. Alternatively, the DPB 4005 may be located in a memoryelement separate from the other buffer memories 4001, 4002, and 4003.The DPB 4005 temporarily stores the decoded pictures. When a P pictureor B picture is to be decoded by the DEC 4004, the DPB 4005 retrievesreference pictures, in response to an instruction from the DEC 4004,from among stored, decoded pictures. The DPB 4005 then provides thereference pictures to the DEC 4004. Furthermore, the DPB 4005 writes thestored pictures into the primary video plane memory 4090 at the timeshown by the PTSs included in the original PES packets.

The secondary video decoder 4071 includes the same structure as theprimary video decoder 4070. The secondary video decoder 4071 firstdecodes the TS packets of the secondary video stream received from thePID filter 4040 into uncompressed pictures. Subsequently, the secondaryvideo decoder 4071 writes the uncompressed pictures into the secondaryvideo plane memory 4091 at the time shown by the PTSs included in thePES packets.

The PG decoder 4072 decodes the TS packets received from the PID filter4040 into uncompressed graphics data and writes the uncompressedgraphics data to the PG plane memory 4092 at the time shown by the PTSsincluded in the PES packets

FIG. 41A is a flowchart of processing whereby the PG decoder 4072decodes a graphics object from one data entry in the PG stream. Theprocessing is started when the PG decoder 4072 receives a group of TSpackets constituting one data entry shown in FIG. 6, from the PID filter4040. FIGS. 41B-41E are schematic diagrams showing the graphics objectchanging as the processing proceeds.

In step S4101, the PG decoder 4072 first identifies an ODS having thesame object ID as the reference object ID 605 in the PCS. Next, the PGdecoder 4072 decodes a graphics object from the identified ODS, andwrites the decoded graphics object into the object buffer. Here, the“object buffer” is a buffer memory embedded in the PG decoder 4072. The“smile mark” FOB shown in FIG. 41B is an example of the graphics objectwritten into the object buffer.

In step S4102, the PG decoder 4072 performs the cropping process inaccordance with the cropping information 602 in the PCS, extracts a partof the graphics object from the graphics object, and writes theextracted part into the object buffer. FIG. 41C shows that strips LSTand RST are removed from the left-hand and right-hand ends of the smilemark FOB, and the remaining part OBJ is written into the object buffer.

In step S4103, the PG decoder 4072 first identifies a WDS having thesame window ID as the reference window ID 603 in the PCS. Next, the PGdecoder 4072 determines a display position of the graphics object in thegraphics plane from a window position 612 indicated by the identifiedWDS and an object display position 601 in the PCS. In FIG. 41D, theupper-left position of the window WIN in the graphics plane GPL and anupper-left position DSP of the graphics object OBJ are determined.

In step S4104, the PG decoder 4072 writes the graphics object in theobject buffer into the display position determined in step S4103. Whendoing so, the PG decoder 4072 determines a range in which the graphicsobject is rendered by using a window size 613 indicated by the WDS. InFIG. 41D, the graphics object OBJ is written into the graphics plane GPLin the range of window WIN starting from the upper-left position DSP.

In step S4105, the PG decoder 4072 first identifies a PDS having thesame pallet ID as the reference pallet ID 604 in the PCS. Next, the PGdecoder 4072, by using CLUT 622 in the PDS, determines color coordinatevalues to be indicated by each pixel data in the graphics object OBJ. InFIG. 41E, the color of each pixel in the graphics object OBJ have beendetermined. In this way, processing of rendering a graphics objectincluded in one data entry is completed. Steps S4101-S4105 are executedby the time indicated by the PTS included in the same PES packet as thegraphics object.

The IG decoder 4073 decodes the TS packets received from the PID filter4040 into uncompressed graphics object. The IG decoder 4073 furtherwrites the uncompressed graphics object to the IG plane memory 4093 atthe time shown by the PTSs included in the PES packets restored from theTS packets. Details on these processes are the same as in the PG decoder4072.

The primary audio decoder 4074 first stores the TS packets received fromthe PID filter 4040 in a buffer provided therein. Subsequently, theprimary audio decoder 4074 removes the TS header and the PES header fromeach TS packet in the buffer, and decodes the remaining data intouncompressed LPCM audio data. Furthermore, the primary audio decoder4074 transmits the resultant audio data to the audio mixer 4095 at thetime shown by the PTS included in the original PES packet. The primaryaudio decoder 4074 selects the decoding method for compressed audio datain accordance with the compression encoding method and stream attributesfor the primary audio stream included in the TS packets. Compressionencoding methods include, for example, AC-3 and DTS.

The secondary audio decoder 4075 has the same structure as the primaryaudio decoder 4074. The secondary audio decoder 4075 first reconstructsPES packets from the TS packets of the secondary audio stream receivedfrom the PID filter 4040 and then decodes the data included in the PESpayloads into uncompressed LPCM audio data. Subsequently, the secondaryaudio decoder 4075 transmits the uncompressed LPCM audio data to theaudio mixer 4095 at the times shown by the PTSs included in the PESheaders. The secondary audio decoder 4075 selects the decoding methodfor compressed audio data in accordance with the compression encodingmethod and stream attributes for the secondary audio stream included inthe TS packets. Compression encoding methods include, for example, DolbyDigital Plus and DTS-HD LBR.

The audio mixer 4095 receives uncompressed audio data from both theprimary audio decoder 4074 and the secondary audio decoder 4075 and thenmixes the received data. The audio mixer 4095 also transmits thesynthesized sound yielded by mixing audio data to, for example, aninternal speaker 103A of the display device 103.

The image processor 4080 receives graphics data, i.e., PNG or JPEGraster data, from the program execution unit 3734. Upon receiving thegraphics data, the image processor 4080 renders the graphics data andwrites the graphics data to the image plane memory 4094.

<Structure of 3D Playback Device>

When playing back 3D video content from the BD-ROM disc 101 in 3Dplayback mode, the playback device 102 operates as a 3D playback device.The fundamental part of the device's structure is identical to the 2Dplayback device shown in FIGS. 37 and 40. Therefore, the following is adescription on sections of the structure of the 2D playback device thatare enlarged or modified. Details on the fundamental parts of the 3Dplayback device can be found in the above description on the 2D playbackdevice. The 3D playback device also uses the same structure as the 2Dplayback device for 2D playlist playback processing. Accordingly, thedetails on this structure can be found in the description on the 2Dplayback device. The following description assumes playback processingof 3D video images in accordance with 3D playlist files, i.e. 3Dplaylist playback processing.

FIG. 42 is a functional block diagram of a 3D playback device 4200. The3D playback device 4200 includes a BD-ROM drive 4201, playback unit4202, and control unit 4203. The playback unit 4202 includes a switch4220, first read buffer (hereinafter, abbreviated as RB1) 4221, secondread buffer (hereinafter, abbreviated as RB2) 4222, system targetdecoder 4225, plane adder 4226, and HDMI communication unit 4227. Thecontrol unit 4203 includes a dynamic scenario memory 4231, staticscenario memory 4232, user event processing unit 4233, program executionunit 4234, playback control unit 4235, and player variable storage unit4236. The playback unit 4202 and the control unit 4203 are eachimplemented on a different integrated circuit, but may alternatively beimplemented on a single integrated circuit. In particular, the dynamicscenario memory 4231, static scenario memory 4232, user event processingunit 4233, and program execution unit 4234 have an identical structurewith the 2D playback device shown in FIG. 37. Accordingly, detailsthereof can be found in the above description on the 2D playback device.

When instructed by the program execution unit 4234 or other unit toperform 3D playlist playback processing, the playback control unit 4235reads a PI from the 3D playlist file stored in the static scenariomemory 4232 in order, setting the read PI as the current PI. Each timethe playback control unit 4235 sets a current PI, it sets operationconditions on the system target decoder 4225 and the plane adder 4226 inaccordance with the STN table of the PI and the STN table SS in the 3Dplaylist file. Specifically, the playback control unit 4235 selects thePID of the elementary stream for decoding and transmits the PID,together with the attribute information necessary for decoding theelementary stream, to the system target decoder 4225. If a PG stream orIG stream is included in the elementary stream indicated by the selectedPID, the playback control unit 4235 specifies the reference offset ID3201 and offset adjustment value 3202 allocated to the stream data,setting the reference offset ID 3201 and offset adjustment value 3202 tothe SPRM(27) and SPRM(28) in the player variable storage unit 4236. Theplayback control unit 4235 also selects the presentation mode of eachpiece of plane data in accordance with the offset during pop-up 3311indicated by the STN table SS, indicating the selected presentation modeto the system target decoder 4225 and plane adder 4226.

Next, in accordance with the current PI, the playback control unit 4235indicates the range of the LBNs in the sector group recorded in theextent SS to be read to the BD-ROM drive 4201 via the procedures in thedescription on FIG. 24E. Meanwhile, the playback control unit 4235refers to the extent start points in the clip information file stored inthe static scenario memory 4232 to generate information indicating theboundary of the data blocks in each extent SS. This informationindicates, for example, the number of source packets from the top of theextent SS to each boundary. The playback control unit 4235 thentransmits this information to the switch 4220.

The player variable storage unit 4236 includes the SPRMs shown in FIG.38, like the player variable storage unit 3736 in the 2D playbackdevice. However, unlike FIG. 38, SPRM(24) includes the first flag, andSPRM(25) includes the second flag, as shown in FIG. 36. In this case,when the SPRM(24) is “0”, the playback device 102 only supports playbackof 2D video images, and when the SPRM(24) is “1”, the playback device102 also supports playback of 3D video images. The playback device 102is in L/R mode when the SPRM(25) is “0” and is in depth mode when theSPRM(25) is “1”. Furthermore, the playback device 102 is in 2D playbackmode when the SPRM(25) is “2”.

Furthermore, in the player variable storage unit 4236, unlike FIG. 38,the SPRM(27) includes a storage area for a reference offset ID for eachgraphics plane, and the SPRM(28) includes a storage area for an offsetadjustment value for each graphics plane. FIG. 43 is a table showing adata structure of SPRM(27) and SPRM(28). As shown in FIG. 43, SPRM(27)includes an area for storing four types of reference offset IDs4310-4313. These reference offset IDs 4310, 4311, 4312, and 4313 arerespectively for a PG plane (PG_ref_offset_id), IG plane(IG_ref_offset_id), secondary video plane (SV_ref_offset_id), and imageplane (IM_ref_offset_id). The SPRM(28) includes an area for storing fourtypes of offset adjustment values 4320-4323. These offset adjustmentvalues 4320, 4321, 4322, and 4323 are respectively for a PG plane(PG_offset_adjustment), IG plane (IG_offset_adjustment), secondary videoplane (SV_offset_adjustment), and image plane (IM_offset_adjustment).

The BD-ROM drive 4201 includes the same components as the BD-ROM drive3701 in the 2D playback device shown in FIG. 37. Upon receiving anindication from the playback control unit 4235 of a range of LBNs, theBD-ROM drive 4201 reads data from the sectors on the BD-ROM disc 101 asindicated by the range. In particular, a source packet group belongingto an extent in the file SS, i.e. belonging to an extent SS, aretransmitted from the BD-ROM drive 4201 to the switch 4220. Each extentSS includes one or more pairs of a base-view and dependent-view datablock, as shown in FIG. 19. These data blocks have to be transferred inparallel to different RB1 4221 and RB2 4222. Accordingly, the BD-ROMdrive 4201 is required to have at least the same access speed as theBD-ROM drive 3701 in the 2D playback device.

The switch 4220 receives an extent SS from the BD-ROM drive 4201. On theother hand, the switch 4220 receives, from the playback control unit4235, information indicating the boundary in each data block included inthe extent SS, i.e. the number of source packets from the top of theextent SS to each boundary. The switch 4220 then refers to thisinformation (i) to extract base-view extents from each extent SS andtransmit the extents to the RB1 4221, and (ii) to extract dependent-viewextents and transmit the extents to the RB2 4222.

The RB1 4221 and RB2 4222 are buffer memories that use a memory elementin the playback unit 4202. In particular, different areas in a singlememory element are used as the RB1 4221 and RB2 4222. Alternatively,different memory elements may be used as the RB1 4221 and RB2 4222. TheRB1 4221 receives base-view extents from the switch 4220 and storesthese extents. The RB2 4222 receives dependent-view extents from theswitch 4220 and stores these extents.

In 3D playlist playback processing, the system target decoder 4225 firstreceives PIDs for stream data to be decoded, as well as attributeinformation necessary for decoding the stream data, from the playbackcontrol unit 4235. The system target decoder 4225 then reads sourcepackets alternately from base-view extents stored in the RB1 4221 anddependent-view extents stored in the RB2 4222. Next, the system targetdecoder 4225 separates, from each source packet, elementary streamsindicated by the PIDs received from the playback control unit 4235 anddecodes the elementary streams. The system target decoder 4225 thenwrites the decoded elementary streams in internal plane memory accordingto the type thereof. The base-view video stream is written in theleft-video plane memory, and the dependent-view video stream is writtenin the right-video plane memory. On the other hand, the secondary videostream is written in the secondary video plane memory, the IG stream inthe IG plane memory, and the PG stream in the PG plane memory. When thesecondary video stream is composed of a pair of a base-view and adependent-view video stream, separate secondary video plane memories areprepared for both the left-view and right-view pieces of plane data. Thesystem target decoder 4225 additionally renders graphics data from theprogram execution unit 4234, such as JPEG, PNG, etc. raster data, andwrites this data in the image plane memory.

The system target decoder 4225 associates the output mode of plane datafrom the left-video and right-video plane memories with B-D presentationmode and B-B presentation mode as follows. When the playback controlunit 4235 indicates B-D presentation mode, the system target decoder4225 alternately outputs plane data from the left-video and right-videoplane memories. On the other hand, when the playback control unit 4235indicates B-B presentation mode, the system target decoder 4225 outputsplane data from only the left-video or right-video plane memory twiceper frame while maintaining the operation mode in 3D playback mode.

When the playback control unit 4235 indicates 1 plane+offset mode, theneach time the system target decoder 4225 reads the VAU at the top ofeach video sequence from the dependent-view video stream, the systemtarget decoder 4225 reads the offset metadata 1110 from the VAU. In theplayback section of the video sequence, the system target decoder 4225first specifies the PTS stored in the same PES packet along with eachVAU and specifies the number of the frame represented by the compressedpicture data of the VAU. The system target decoder 4225 then reads theoffset information associated with the frame number from the offsetmetadata and transmits the offset information to the plane adder 4226 atthe time indicated by the specified PTS.

The plane adder 4226 receives each type of plane data from the systemtarget decoder 4225 and superimposes these pieces of plane data on oneanother to create one combined frame or field. In particular, in L/Rmode, left-video plane data represents a left-view video plane, andright-view plane data represents a right-view video plane. Accordingly,the plane adder 4226 superimposes other plane data representing the leftview on the left-video plane data and superimposes other plane datarepresenting the right view on the right-video plane data. On the otherhand, in depth mode, the right-video plane data represents a depth mapfor the video plane representing the left-video plane data. Accordingly,the plane adder 4226 first generates a pair of left-view and right-viewpieces of video plane data from the corresponding pieces of video planedata. Subsequently, the plane adder 4226 performs the same combinationprocessing as in L/R mode.

When receiving an indication of 1 plane+offset mode or 1 plane+zerooffset mode from the playback control unit 4235 as the presentation modefor the secondary video plane, PG plane, IG plane, or image plane, theplane adder 4226 performs offset control on the plane data received fromthe system target decoder 4225. A pair of left-view plane data andright-view plane data is thus generated.

In particular, when 1 plane+offset mode is indicated, the plane adder4226 first reads one of the reference offset IDs 4310-4313 thatcorresponds to each graphics plane from the SPRM(27) in the playervariable storage unit 4236. Next, the plane adder 4226 refers to theoffset information received from the system target decoder 4225 toretrieve offset information, namely an offset direction 1122 and offsetvalue 1123, belonging to the offset sequence 1113 indicated by eachreference offset ID 4310-4313. Subsequently, the plane adder 4226 readsone of the offset adjustment values 4320-4323 that corresponds to eachgraphics plane from the SPRM(28) in the player variable storage unit4236 and adds each offset adjustment value to the corresponding offsetvalue. The plane adder 4226 then uses each offset value to performoffset control on the corresponding graphics plane.

On the other hand, when 1 plane+zero offset mode is indicated, the planeadder 4226 does not refer to either SPRM(27) or SPRM(28), but ratherperforms offset control on each graphics plane with an offset value of“0”. Accordingly, the same plane data is used for both the left-view andright-view graphics planes and combined with other pieces of plane data.

The HDMI communication unit 4227, connected with the display device 103via an HDMI cable 122, exchanges CEC messages with the display device103 via the HDMI cable 122. This causes the HDMI communication unit 4227to perform an HDMI authentication of the display device 103 and ask thedisplay device 103 whether or not playback of 3D video images issupported.

<<3D Playlist Playback Processing>>

FIG. 44 is a flowchart of 3D playlist playback processing by a playbackcontrol unit 4235. 3D playlist playback processing is started by theplayback control unit 4235 reading a 3D playlist file from the staticscenario memory 4232.

In step S4401, the playback control unit 4235 first reads a single PIfrom a main path in the 3D playlist file and then sets the PI as thecurrent PI. Next, from the STN table of the current PI, the playbackcontrol unit 4235 selects PIDs of elementary streams to be played backand specifies attribute information necessary for decoding theelementary streams. The playback control unit 4235 further selects, fromamong the elementary streams corresponding to the current PI in the STNtable SS in the 3D playlist file, a PID of elementary streams that areto be added to the elementary streams to be played back, and playbackcontrol unit 4235 specifies attribute information necessary for decodingthese elementary streams. The selected PIDs and attribute informationare indicated to the system target decoder 4225. The playback controlunit 4235 additionally specifies, from among sub-paths in the 3Dplaylist file, a SUB_PI to be referenced at the same time as the currentPI, specifying this SUB_PI as the current SUB_PI. Thereafter, processingproceeds to step S4402.

In step S4402, the playback control unit 4235 selects the display modefor each piece of plane data based on the offset during pop-up indicatedby the STN table SS and indicates the display mode to the system targetdecoder 4225 and the plane adder 4226. In particular, when the value ofthe offset during pop-up is “0”, B-D presentation mode is selected asthe video plane presentation mode, and 1 plane+offset mode is selectedas the presentation mode for the graphics plane. On the other hand, whenthe value of the offset during pop-up is “1”, B-B presentation mode isselected as the video plane presentation mode, and 1 plane+zero offsetmode is selected as the presentation mode for the graphics plane.Thereafter, processing proceeds to step S4403.

In step S4403, the playback control unit 4235 checks whether 1plane+offset mode or 1 plane+zero offset mode has been selected as thepresentation mode of the graphics plane. If 1 plane+offset mode has beenselected, processing proceeds to step S4404. If 1 plane+zero offset modehas been selected, processing proceeds to step S4405.

In step S4404, the playback control unit 4235 refers to the STN table ofthe current PI and retrieves the PG stream, IG stream, or text subtitlestream from among the elementary streams indicated by the selected PIDs.Furthermore, the playback control unit 4235 specifies the referenceoffset ID and offset adjustment value allocated to the pieces of streamdata, setting the reference offset ID and offset adjustment value to theSPRM(27) and SPRM(28) in the player variable storage unit 4236.Thereafter, processing proceeds to step S4405.

In step S4405, the playback control unit 4235 reads reference clipinformation, a PTS #1 indicating a playback start time IN1, and a PTS #2indicating a playback end time OUT1 from the current PI and the SUB_PI.From this reference clip information, a clip information filecorresponding to each of the file 2D and the file DEP to be played backis specified. Thereafter, processing proceeds to step S4406.

In step S4406, with reference to the entry map in each of the clipinformation files specified in step S4405, the playback control unit4235 retrieves the SPN #1 and SPN #2 in the file 2D, and the SPN #11 andSPN #12 in the file DEP, corresponding to the PTS #1 and the PTS #2. Asdescribed with reference to FIG. 24, referring to extent start points ofeach clip information file, the playback control unit 4235 furthercalculates, from the SPN #1 and the SPN #11, the number of sourcepackets SPN #21 from the top of the file SS to the playback startposition. The playback control unit 4235 also calculates, from the SPN#2 and the SPN #12, the number of source packets SPN #22 from the top ofthe file SS to the playback end position. Specifically, the playbackcontrol unit 4235 first retrieves, from among SPNs shown by extent startpoints of the 2D clip information files, a value “Am” that is thelargest value less than or equal to SPN #1, and retrieves, from amongthe SPNs shown by extent start points of dependent-view clip informationfiles, a value “Bm” that is the largest value less than or equal to theSPN #11. Next, the playback control unit 4235 obtains the sum of theretrieved SPNs Am+Bm and sets the sum as SPN #21. Next, the playbackcontrol unit 4235 retrieves, from among SPNs shown by the extent startpoints of the 2D clip information files, a value “An” that is thesmallest value that is larger than the SPN #2. The playback control unit4235 also retrieves, from the SPNs of the extent start points of thedependent-view clip information files, a value “Bn” that is the smallestvalue that is larger than the SPN #12. Next, the playback control unit4235 obtains the sum of the retrieved SPNs An+Bn and sets the sum as SPN#22. Thereafter, processing proceeds to step S4407.

In step S4407, the playback control unit 4235 converts the SPN #21 andthe SPN #22, determined in step S4406, into a pair of numbers of sectorsN1 and N2. Specifically, the playback control unit 4235 first obtainsthe product of SPN #21 and the data amount per source packet, i.e. 192bytes. Next, the playback control unit 4235 divides this product thedata amount per sector, i.e. 2048 bytes: SPN #21×192/2048. The resultingquotient is the same as the number of sectors N1 from the top of thefile SS to immediately before the playback start position. Similarly,from the SPN #22, the playback control unit 4235 calculates SPN#22×192/2048. The resulting quotient is the same as the number ofsectors N2 from the top of the file SS to immediately before theplayback end position. Thereafter, processing proceeds to step S4408.

In step S4408, the playback control unit 4235 specifies, from thenumbers of sectors N1 and N2 obtained in step S4407, LBNs of the top andend of the extent SS group to be played back. Specifically, withreference to the file entry of the file SS to be played back, theplayback control unit 4235 counts from the top of sector group in whichthe extent SS group is recorded so that the LBN of the (N1+1)^(th)sector=LBN #1, and the LBN of the (N2+1)^(th) sector=LBN #2. Theplayback control unit 4235 further specifies a range from the LBN#1 tothe LBN#2 to the BD-ROM drive 4201. As a result, from the sector groupin the specified range, a source packet group belonging to an extent SSgroup is read in aligned units. Thereafter, processing proceeds to stepS4409.

In step S4409, referring to the extent start points of the clipinformation file used in step S4406, the playback control unit 4235generates information (hereinafter referred to as “data block boundaryinformation”) indicating a boundary between dependent-view blocks andbase-view data blocks included in the extent SS group, transmitting thedata block boundary information to the switch 4220. As a specificexample, assume that the SPN #21 indicating the playback start positionis the same as the sum of SPNs indicating the extent start points,An+Bn, and that the SPN#22 indicating the playback end position is thesame as the sum of SPNs indicating the extent start points, Am+Bm. Inthis case, the playback control unit 4235 obtains a sequence ofdifferences between SPNs from the respective extent start points,A(n+1)−An, B(n+1)−Bn, A(n+2)−A(n+1), B(n+2)−B(n+1), . . . , Am−A(m−1),and Bm−B(m−1), and transmits the sequence to the switch 4220 as the datablock boundary information. As shown in FIG. 24E, this sequenceindicates the number of source packets of data blocks included in theextent SS. The switch 4220 counts, from zero, the number of sourcepackets of the extents SS received from the BD-ROM drive 4201. Each timethe count is the same as the difference between SPNs indicated by thedata block boundary information, the switch 4220 switches thedestination of output of the source packets between the RB1 4221 and theRB2 4222 and resets the count to zero. As a result, {B(n+1)−Bn} sourcepackets from the top of the extent SS are output to the RB2 4222 as thefirst dependent-view extent, and the following {A(n+1)−An} sourcepackets are transmitted to the RB1 4221 as the first base-view extent.Thereafter, dependent-view extents and base-view extents are extractedfrom the extent SS alternately in the same way, alternating each timethe number of source packets received by the switch 4220 is the same asthe difference between SPNs indicated by the data block boundaryinformation.

In step S4410, the playback control unit 4235 checks whether anunprocessed PI remains in the main path. When an unprocessed PI remains,processing is repeated from step S4401. When no unprocessed PI remains,processing ends.

<<System Target Decoder>>

The following two means are conceivable as specific means used by thesystem target decoder 4225 to implement the function to extract offsetmetadata from the dependent-view video stream. The first meansincorporates a TS priority filter and an offset metadata processing unitinto the system target decoder 4225 as modules separate from the primaryvideo decoder. The TS priority filter selects TS packets containingoffset metadata and TS packets containing dependent-view pictures,depending on the values of TS priority flags. The offset metadataprocessing unit extracts offset information from the TS packetscontaining offset metadata. The second means causes TS packetscontaining a dependent-view video stream to be sent to the primary videodecoder in the system target decoder 4225, regardless of the values ofTS priority flags. The primary video decoder extracts offset informationfrom the dependent-view video stream in parallel with the process ofdecoding the dependent-view video stream.

(First Means)

FIG. 45 is a functional block diagram of the system target decoder 4225by using the first means. The components shown in FIG. 45 differ fromthe components of the system target decoder 3724 in the 2D playbackdevice shown in FIG. 40 as follows: (A) the input system from the readbuffers to the decoders has a duplex configuration; and (B) the TSpriority filter and the offset metadata processing unit are provided.The primary audio decoder, secondary audio decoder, audio mixer, imageprocessor, and plane memories have similar structures to those in the 2Dplayback device shown in FIG. 40. Accordingly, among the componentsshown in FIG. 45, ones different from the components shown in FIG. 40are described below. On the other hand, details of similar componentscan be found in the description on FIG. 40. Furthermore, since the videodecoders each have a similar structure, the structure of the primaryvideo decoder 4515 is described below. This description is also validfor the structures of other video decoders.

The first source depacketizer 4511 reads source packets from the firstread buffer 4221. The first source depacketizer 4511 further retrievesTS packets included in the source packets and transmits the TS packetsto the first PID filter 4513. The second source depacketizer 4512 readssource packets from the second read buffer 4222, furthermore retrievingTS packets included in the source packets and transmitting the TSpackets to the second PID filter 4514. Each of the source depacketizers4511 and 4512 further synchronizes the time of transferring the TSpackets with the time shown by the ATS of each source packet. Thissynchronization method is the same method as the source depacketizer4010 shown in FIG. 40. Accordingly, details thereof can be found in thedescription provided for FIG. 40. With this sort of adjustment oftransfer time, the mean transfer rate of TS packets from the firstsource depacketizer 4511 to the first PID filter 4513 does not exceedthe system rate R_(TS1) indicated by the 2D clip information file.Similarly, the mean transfer rate of TS packets from the second sourcedepacketizer 4512 to the second PID filter 4514 does not exceed thesystem rate R_(TS2) indicated by the dependent-view clip informationfile.

The first PID filter 4513 compares the PID of each TS packet receivedfrom the first source depacketizer 4511 with the selected PID. Theplayback control unit 4235 designates the selected PID beforehand inaccordance with the STN table in the 3D playlist file. When the two PIDsmatch, the first PID filter 4513 transfers the TS packets to the decoderassigned to the PID. For example, if a PID is 0x1011, the TS packets aretransferred to TB1 4501 in the primary video decoder 4515. On the otherhand, TS packets with PIDs ranging from 0x1B00-0x1B1F, 0x1100-0x111F,0x1A00-0x1A1F, 0x1200-0x121F, and 0x1400-0x141F are transferred to thesecondary video decoder, primary audio decoder, secondary audio decoder,PG decoder, and IG decoder, respectively.

The second PID filter 4514 compares the PID of each TS packet receivedfrom the second source depacketizer 4512 with the selected PID. Theplayback control unit 4235 designates the selected PID beforehand inaccordance with the STN table SS in the 3D playlist file. When the twoPIDs match, the second PID filter 4514 transfers the TS packets to thedecoder assigned to the PID or the TS priority filter 4551. For example,if a PID is 0x1012 or 0x1013, the TS packets are transferred to the TSpriority filter 4551. On the other hand, TS packets with PIDs rangingfrom 0x1B20-0x1B3F, 0x1220-0x127F, and 0x1420-0x147F are transferred tothe secondary video decoder, PG decoder, and IG decoder, respectively.

The TS priority filter 4551 receives TS packets from the second PIDfilter 4514 and reads TS priority 511 from the TS header 501H in each ofthe TS packets. Here, TS packets with PID=0x1012 or 0x1013 aretransferred from the second PID filter 4514 to the TS priority filter4551. These TS packets contain a dependent-view video stream.

Among the TS packets in the sequence 1520 shown in FIG. 15, the firstgroup 1521 and the third group 1523 have the TS priority of “0,” and thesecond group 1522 has the TS priority of “1”. The TS priority filter4551 transfers TS packets with TS priority=0 from the sequence 1520 tothe TB2 4508 in the primary video decoder 4515, and TS packets with TSpriority=1 to the offset metadata processing unit 4552. As shown in FIG.15, the TS packets with TS priority=1 belong to the second group 1522.Accordingly, the TS payloads thereof include only the supplementary data1504 consisting only of the offset metadata 1509. As a result, among theVAU #1 in the dependent-view video stream, supplementary data consistingonly of the offset metadata 1509 is transferred to the offset metadataprocessing unit 4552, and the remaining data, which include othersupplementary data, are transferred to the primary video decoder 4515.

Among the TS packets in the sequence 1620 shown in FIG. 16, the firstgroup 1621 and the second group 1622 have the TS priority of “l,” andthe third group 1623 has the TS priority of “0”. The TS priority filter4551 transfers TS packets with TS priority=0 from the sequence 1620 tothe TB2 4508 in the primary video decoder 4515, and TS packets with TSpriority=1 to both the TB2 4508 and the offset metadata processing unit4552. Accordingly, VAU #1 in the dependent-view video stream istransferred to the primary video decoder 4515, while the elements fromthe sub-AU identification code to the supplementary data are transferredto the offset metadata processing unit 4552 as well.

The primary video decoder 4515 includes a TB1 4501, MB1 4502, EB1 4503,TB2 4508, MB2 4509, EB2 4510, buffer switch 4506, DEC 4504, DPB 4505,and picture switch 4507. The TB1 4501, MB1 4502, EB1 4503, TB2 4508, MB24509, EB2 4510 and DPB 4505 are all buffer memories. Each of thesebuffer memories uses an area of a memory element included in the primaryvideo decoder 4515. Alternatively, some or all of these buffer memoriesmay be separated on different memory elements.

The TB1 4501 receives TS packets that include a base-view video streamfrom the first PID filter 4513 and stores the TS packets as they are.The MB1 4502 stores PES packets reconstructed from the TS packets storedin the TB1 4501. The TS headers of the TS packets are removed at thispoint. The EB1 4503 extracts and stores encoded VAUs from the PESpackets stored in the MB1 4502. The PES headers of the PES packets areremoved at this point.

The TB2 4508 receives TS packets that include the dependent-view videostream from the TS priority filter 4551 and stores the TS packets asthey are. The MB2 4509 stores PES packets reconstructed from the TSpackets stored in the TB2 4508. The TS headers of the TS packets areremoved at this point. The EB2 4510 extracts and stores encoded VAUsfrom the PES packets stored in the MB2 4509. The PES headers of the PESpackets are removed at this point.

The buffer switch 4506 transfers the headers of the VAUs stored in theEB1 4503 and the EB2 4510 in response to a request from the DEC 4504.Furthermore, the buffer switch 4506 transfers the compressed picturedata for the VAUs to the DEC 4504 at the times indicated by the DTSsincluded in the original PES packets. In this case, the DTSs are equalbetween a pair of pictures belonging to the same 3D VAU between thebase-view video stream and dependent-view video stream. Accordingly, fora pair of VAUs that have the same DTS, the buffer switch 4506 firsttransmits the VAU stored in the EB1 4503 to the DEC 4504. Additionally,the buffer switch 4506 may cause the DEC 4504 to return the decodingswitch information 1750 in the VAU. In such a case, the buffer switch4506 can determine if it should transfer the next VAU from the EB1 4503or the EB2 4510 by referring to the decoding switch information 1750.

Like the DEC 4004 shown in FIG. 40, the DEC 4504 is a hardware decoderspecifically for decoding of compressed pictures and is composed of anLSI that includes, in particular, a function to accelerate the decoding.The DEC 4504 decodes the compressed picture data transferred from thebuffer switch 4506 in order. During decoding, the DEC 4504 firstanalyzes each VAU header to specify the compressed picture, compressionencoding method, and stream attribute stored in the VAU, selecting adecoding method in accordance with this information. Compressionencoding methods include, for example, MPEG-2, MPEG-4 AVC, MVC, and VC1.Furthermore, the DEC 4504 transmits the decoded, uncompressed picture tothe DPB 4505.

The DPB 4505 temporarily stores the uncompressed pictures decoded by theDEC 4504. When the DEC 4504 decodes a P picture or a B picture, the DPB4505 retrieves reference pictures from among the stored, uncompressedpictures in response to a request from the DEC 4504 and supplies theretrieved reference pictures to the DEC 4504.

The picture switch 4507 writes the uncompressed pictures from the DPB4505 to either the left-video plane memory 4520 or the right-video planememory 4521 at the time indicated by the PTS included in the originalPES packet. In this case, the PTSs are equal between a base-view pictureand a dependent-view picture belonging to the same 3D VAU. Accordingly,for a pair of pictures that have the same PTS and that are stored by theDPB 4505, the picture switch 4507 first writes the base-view picture inthe left-video plane memory 4520 and then writes the dependent-viewpicture in the right-video plane memory 4521.

The offset metadata processing unit 4552 is implemented on the same chipas the primary video decoder 4515, but configured as a module separatefrom the primary video decoder 4515. Alternatively, the offset metadataprocessing unit 4552 may be implemented on a chip separate from the chipon which the primary video decoder 4515 is implemented. Furthermore, theoffset metadata processing unit 4552 may be configured as dedicatedhardware or realized as general-purpose hardware controlled by software.The offset metadata processing unit 4552 analyzes TS packets transferredfrom the TS priority filter 4551, and then reads offset metadata fromsupplementary data stored in the TS payloads of the TS packets.

The sequence of TS packets 1520 shown in FIG. 15 contains the PES headerbelonging to the same PES packet as VAU #1 in the group of TS packets tobe transferred to the primary video decoder 4515. Accordingly, theoffset metadata processing unit 4552 reads the PTS of the framerepresented by the VAU #1 from the offset metadata. On the other hand,the sequence of TS packets 1620 shown in FIG. 16 contains the PES headerin the group of TS packets to be transferred to the offset metadataprocessing unit 4552, as well as the group of TS packets to betransferred to the primary video decoder 4515. Accordingly, the offsetmetadata processing unit 4552 may read the PTS of the frame representedby the VAU #1 from either of the PES header and the offset metadata.

The offset metadata processing unit 4552 increments the frame number by1 at frame intervals, starting from 0 at the time indicated by the PTS.In synchronization with the incrementing action, the offset metadataprocessing unit 4552 further retrieves offset information associatedwith each frame number from the offset metadata, and then transmits theoffset information to the plane adder 4226. Here, the TS priority filter4551 prevents compressed picture data from being transferred from eitherof the sequences of TS packets 1520 and 1620 shown in FIGS. 15 and 16 tothe offset metadata processing unit 4552. Accordingly, the offsetmetadata processing unit 4552 can reliably manage the offset informationwithout interference from the compressed picture data.

(Second Means)

FIG. 46 is a functional block diagram showing a system for processingvideo streams, the system included in the system target decoder 4225that uses the second means. The system target decoder 4225 shown in FIG.46 differs from that shown in FIG. 45 in the function of the DEC 4604 inthe primary video decoder 4614. Other components are similar tocorresponding ones. In FIG. 46, components similar to ones shown in FIG.45 are marked with the same reference numbers. Furthermore, details ofthe similar components can be found in the description on FIG. 45.

Like the DEC 4504 shown in FIG. 45, the DEC 4604 is a hardware decoderspecifically for decoding of compressed pictures, and is composed of anLSI that includes, in particular, a function to accelerate the decoding.The DEC 4604 decodes the compressed picture data transferred from thebuffer switch 4506 in order, and transfers the decoded uncompressedpictures to the DPB 4505. Furthermore, each time it reads a VAU locatedat the top of each video sequence from the dependent-view video stream,the DEC 4604 reads offset metadata from the VAU. In the playback sectionof the video sequence, the DEC 4604 first identifies the PTS which isstored in the same PES packet as the VAU, and the frame numberrepresented by the compressed picture data of the VAU. Next, the DEC4604 reads offset information associated with the frame number from theoffset metadata, and sends the offset information to the plane adder4226 at the time indicated by the identified PTS.

<<Plane Adders>>

FIG. 47 is a functional block diagram of a plane adder 4226. As shown inFIG. 47, the plane adder 4226 includes a parallax video generation unit4710, switch 4720, four cropping units 4731-4734, and four adders4741-4744.

The parallax video generation unit 4710 receives left-video plane data4701 and right-video plane data 4702 from the system target decoder4225. In the playback device 102 in L/R mode, the left-video plane data4701 represents the left-view video plane, and the right-video planedata 4702 represents the right-view video plane. At this point, theparallax video generation unit 4710 transmits the left-video plane data4701 and the right-video plane data 4702 as they are to the switch 4720.On the other hand, in the playback device 102 in depth mode, theleft-video plane data 4701 represents the video plane for 2D videoimages, and the right-video plane data 4702 represents a depth map forthe 2D video images. In this case, the parallax video generation unit4710 first calculates the binocular parallax for each element in the 2Dvideo images using the depth map. Next, the parallax video generationunit 4710 processes the left-video plane data 4701 to shift thepresentation position of each element in the video plane for 2D videoimages to the left or right according to the calculated binocularparallax. This generates a pair of video planes representing the leftview and right view. Furthermore, the parallax video generation unit4710 transmits the pair of video planes to the switch 4720 as a pair ofpieces of left-video and right-video plane data.

When the playback control unit 4235 indicates B-D presentation mode, theswitch 4720 transmits left-video plane data 4701 and right-video planedata 4702 with the same PTS to the first adder 4741 in that order. Whenthe playback control unit 4235 indicates B-B presentation mode, theswitch 4720 transmits one of the left-video plane data 4701 andright-video plane data 4702 with the same PTS twice per frame to thefirst adder 4741, discarding the other piece of plane data.

The first cropping unit 4731 includes the same structure as a pair ofthe parallax video generation unit 4710 and switch 4720. Thesestructures are used when the secondary video plane data is a pair of aleft view and a right view. In particular, in the playback device 102 indepth mode, the parallax video generation unit in the first croppingunit 4731 converts the secondary video plane data into a pair ofleft-view and right-view pieces of plane data. When the playback controlunit 4235 indicates B-D presentation mode, the left-view and right-viewpieces of plane data are alternately transmitted to the first adder4741. On the other hand, when the playback control unit 4235 indicatesB-B presentation mode, one of the left-view and right-view pieces ofplane data is transmitted twice per frame to the first adder 4741, andthe other piece of plane data is discarded.

When the playback control unit 4235 indicates 1 plane+offset mode, thefirst cropping unit 4731 performs the following offset control on thesecondary video plane data 4703. The first cropping unit 4731 firstreceives offset information 4707 from the system target decoder 4225. Atthis point, the first cropping unit 4731 reads the reference offset ID(SV_ref_offset_id) 4212 corresponding to the secondary video plane fromthe SPRM(27) 4751 in the player variable storage unit 4236. Next, thefirst cropping unit 4731 retrieves the offset information belonging tothe offset sequence indicated by the reference offset ID from the offsetinformation 4707 received from the system target decoder 4225.Subsequently, the first cropping unit 4731 reads the offset adjustmentvalue (SV_offset_adjustment) 4222 corresponding to the secondary videoplane from the SPRM(28) 4752 in the player variable storage unit 4236and adds the offset adjustment value to the retrieved offset value.After that, the first cropping unit 4731 refers to the offset value toperform offset control on the secondary video plane data 4703. As aresult, the secondary video plane data 4703 is converted into a pair ofpieces of secondary video plane data representing a left view and aright view, and this pair is alternately output.

The playback control unit 4235 generally updates the values of theSPRM(27) 4751 and SPRM(28) 4752 each time the current PI changes.Additionally, the program execution unit 4234 may set the values of theSPRM(27) 4751 and the SPRM(28) 4752 in accordance with a movie object orBD-J object.

On the other hand, when the playback control unit 4235 indicates 1plane+zero offset mode, the first cropping unit 4731 does not performoffset control, instead outputting the secondary video plane data 4703twice as is.

Similarly, the second cropping unit 4732 refers to the reference offsetID (PG_ref_offset_id) 4310 for the PG plane and to the offset adjustmentvalue (PG_offset_adjustment) 4320 to perform offset control on the PGplane data 4704. The third cropping unit 4733 refers to the referenceoffset ID (IG_ref_offset_id) 4311 for the IG plane and to the offsetadjustment value (IG_offset_adjustment) 4321 to perform offset controlon the IG plane data 4705. The first cropping unit 4734 refers to thereference offset ID (IM_ref_offset_id) 4213 for the image plane and tothe offset adjustment value (IM_offset_adjustment) 4323 to performoffset control on the image plane data 4706.

[Flowchart of Offset Control]

FIG. 48 is a flowchart of offset control by the cropping units4731-4734. Each of the cropping units 4731-4734 begins offset controlupon receiving offset information 4707 from the system target decoder4225. In the following example, the second cropping unit 4732 performsoffset control on the PG plane data 4704. The other cropping units 4731,4733, and 4734 perform similar processing respectively on the secondaryvideo plane data 4703, IG plane data 4705, and image plane data 4706.

In step S4801, the second cropping unit 4732 first receives PG planedata 4704 from the system target decoder 4225. At this point, the secondcropping unit 4732 reads the reference offset ID (PG_ref_offset_id) 4310for the PG plane from the SPRM(27) 4751. Next, the second cropping unit4731 retrieves the offset information belonging to the offset sequenceindicated by the reference offset ID 4310 from the offset information4707 received from the system target decoder 4225. Thereafter,processing proceeds to step S4802.

In step S4802, the second cropping unit 4732 reads the offset adjustmentvalue (PG_offset_adjustment) 4320 for the PG plane from the SPRM(28)4752 and adds this offset adjustment value to the offset value retrievedin step S4801. Thereafter, processing proceeds to step S4803.

In step S4803, the second cropping unit 4732 checks which of a left viewand a right view is represented by the video plane data selected by theswitch 4720. If the video plane data represents a left view, processingproceeds to step S4804. If the video plane data represents a right view,processing proceeds to step S4807.

In step S4804, the second cropping unit 4732 checks the value of theretrieved offset direction. Hereinafter, the following is assumed: ifthe offset direction value is “0”, the 3D graphics image is closer tothe viewer than the screen, and if the offset direction value is “1”,the image is further back than the screen. In this context, when theoffset direction value is “0”, processing proceeds to step S4805. If theoffset direction value is “1”, processing proceeds to step S4806.

In step S4805, the second cropping unit 4732 provides a right offset tothe PG plane data 4704. In other words, the position of each piece ofpixel data included in the PG plane data 4704 is shifted to the right bythe offset value. Thereafter, processing proceeds to step S4810.

In step S4806, the second cropping unit 4732 provides a left offset tothe PG plane data 4704. In other words, the position of each piece ofpixel data included in the PG plane data 4704 is shifted to the left bythe offset value. Thereafter, processing proceeds to step S4810.

In step S4807, the second cropping unit 4732 checks the value of theretrieved offset direction. If the offset direction value is “0”,processing proceeds to step S4808. If the offset direction value is “1”,processing proceeds to step S4809.

In step S4808, the second cropping unit 4732 provides a left offset tothe PG plane data 4704, contrary to step S4805. In other words, theposition of each piece of pixel data included in the PG plane data 4704is shifted to the left by the offset value. Thereafter, processingproceeds to step S4810.

In step S4809, the second cropping unit 4732 provides a right offset tothe PG plane data 4704, contrary to step S4806. In other words, theposition of each piece of pixel data included in the PG plane data 4704is shifted to the right by the offset value. Thereafter, processingproceeds to step S4810.

In step S4810, the second cropping unit 4732 outputs the processed PGplane data 4704 to the third cropping unit 4734. Processing thenterminates.

[Changes in Plane Data via Offset Control]

FIG. 49B is a schematic diagram showing PG plane data GP to which thesecond cropping unit 4732 is to provide offset control. As shown in FIG.49B, the PG plane data GP includes pixel data representing the subtitle“I love you”, i.e. subtitle data STL. This subtitle data STL is locatedat a distance D0 from the left edge of the PG plane data GP.

FIG. 49A is a schematic diagram showing PG plane data RPG to which aright offset has been provided. As shown in FIG. 49A, when providing aright offset to the PG plane data GP, the second cropping unit 4732changes the position of each piece of pixel data in the PG plane data GPfrom its original position to the right by a number of pixels OFS equalto the offset value. Specifically, the second cropping unit 4732performs cropping to remove, from the right edge of the PG plane dataGP, pixel data included in a strip AR1 of a width OFS equal to theoffset value. Next, the second cropping unit 4732 forms a strip AL1 ofwidth OFS by adding pixel data to the left edge of the PG plane data GP.The pixel data included in this strip AL1 is set as transparent. Thisprocess yields PG plane data RGP to which a right offset has beenprovided. Subtitle data STL is actually located at a distance DR fromthe left edge of this PG plane data RGP. This distance DR equals theoriginal distance D0 plus the offset value OFS: DR=D0+OFS.

FIG. 49C is a schematic diagram showing PG plane data LPG to which aleft offset has been provided. As shown in FIG. 49C, when providing aleft offset to the PG plane data GP, the second cropping unit 4732changes the position of each piece of pixel data in the PG plane data GPfrom its original position to the left by a number of pixels OFS equalto the offset value. Specifically, the second cropping unit 4732performs cropping to remove, from the left edge of the PG plane data GP,pixel data included in a strip AL2 of a width OFS equal to the offsetvalue. Next, the second cropping unit 4732 forms a strip AR2 of widthOFS by adding pixel data to the right edge of the PG plane data GP. Thepixel data included in this strip AR2 is set as transparent. Thisprocess yields PG plane data LGP to which a left offset has beenprovided. Subtitle data STL is actually located at a distance DL fromthe left edge of this PG plane data RGP. This distance DL equals theoriginal distance D0 minus the offset value OFS: DL=D0−OFS.

Referring again to FIG. 47, the first adder 4741 receives video planedata from the switch 4720 and receives secondary video plane data fromthe first cropping unit 4731. At this point, the first adder 4741superimposes each pair of the plane data and secondary video plane dataand transmits the result to the second adder 4742. The second adder 4742receives PG plane data from the second cropping unit 4732, superimposesthis PG plane data on the plane data from the first adder 4741, andtransmits the result to the third adder 4743. The third adder 4743receives IG plane data from the third cropping unit 4733, superimposesthis IG plane data on the plane data from the second adder 4742, andtransmits the result to the fourth adder 4744. The fourth adder 4744receives image plane data from the fourth cropping unit 4734,superimposes this image plane data on the plane data from the thirdadder 4743, and outputs the result to the display device 103. The adders4741-4744 each make use of alpha blending when superimposing plane data.In this way, the secondary video plane data 4703, PG plane data 4704, IGplane data 4705, and image plane data 4706 are superimposed in the ordershown by the arrow 4700 in FIG. 47 on the left-video plane data 4701 orright-video plane data 4702. As a result, the video images indicated byeach piece of plane data are displayed on the screen of the displaydevice 103 so that the left-video plane or right-video plane appears tooverlap with the secondary video plane, IG plane, PG plane, and imageplane in that order.

In addition to the above-stated processing, the plane adder 4724converts the output format of the plane data combined by the four adders4741-4744 into a format that complies with the display method of 3Dvideo images adopted in a device such as the display device 103 to whichthe data is output. If an alternate-frame sequencing method is adoptedin the device, for example, the plane adder 4724 outputs the combinedplane data pieces as one frame or one field. On the other hand, if amethod that uses a lenticular lens is adopted in the device, the planeadder 4724 combines a pair of left-view and right-view pieces of planedata as one frame or one field of video data with use of internal buffermemory. Specifically, the plane adder 4724 temporarily stores and holdsin the buffer memory the left-view plane data that has been combinedfirst. Subsequently, the plane adder 4724 combines the right-view planedata, and further combines the resultant data with the left-view planedata held in the buffer memory. During combination, the left-view andright-view pieces of plane data are each divided, in a verticaldirection, into small rectangular areas that are long and thin, and thesmall rectangular areas are arranged alternately in the horizontaldirection in one frame or one field so as to re-constitute the frame orthe field. In this way, the pair of left-view and right-view pieces ofplane data is combined into one video frame or field. The plane adder4724 then outputs the combined video frame or field to the correspondingdevice.

<Effects of Embodiment 1>

In the BD-ROM disc 101 according to Embodiment 1 of the presentinvention, as shown in FIGS. 15 and 16, TS priorities are assigned to asequence of TS packets which stores a VAU at the top of each videosequence constituting the dependent-view video stream. In particular,different TS priorities are assigned to a TS packet group storing offsetmetadata and a TS packet group storing compressed picture data. In thatcase, the function to extract offset metadata may be realized in thesystem target decoder 4225 by the first means shown in FIG. 45 or thesecond means shown in FIG. 46. The system target decoder 4225 by thefirst means can easily sort out TS packets storing offset metadata fromthe other by using the TS priorities. Accordingly, the primary videodecoder 4515 and the offset metadata processing unit 4552 can beimplemented in different forms. Especially even if the primary videodecoder 4515 is composed of hardware, irrespectively thereof the offsetmetadata processing unit 4552 may be composed of dedicated hardware orrealized by software with use of general-purpose hardware. On the otherhand, in the system target decoder 4225 by the second means, the primaryvideo decoder 4615 can execute in parallel both the function to decodethe dependent-view video stream and the function to extract offsetmetadata. Thus regardless of the TS priorities all TS packets storingthe dependent-view video stream can be passed to the primary videodecoder 4615. In this way, the data structure of the dependent-viewvideo stream and the offset metadata recorded on the BD-ROM disc 101according to Embodiment 1 of the present invention can be used in commonby the system target decoder 4225 realized by the first means and thesystem target decoder 4225 realized by the second means.

<Modifications>

(1-A) Video Stream

In L/R mode according to Embodiment 1 of the present invention, thebase-view video stream represents the left view, and the dependent-viewvideo stream represents the right view. Conversely, however, thebase-view video stream may represent the right view and thedependent-view video stream the left view.

On the BD-ROM disc 101 according to Embodiment 1 of the presentinvention, the base-view video stream and the dependent-view videostream are multiplexed in different TSs. Alternatively, the base-viewvideo stream and the dependent-view video stream may be multiplexed intoa single TS.

(1-B) Offset Metadata

(1-B-1) The offset metadata may be stored in the base-view video streaminstead of in the dependent-view video stream. In this case as well, theoffset metadata is preferably stored in the supplementary data in theVAU located at the top of each video sequence. Furthermore, the 3Dplaylist file may be provided with a flag indicating whether thebase-view video stream or the dependent-view video stream includes theoffset metadata. This allows for an increase in the degree of freedomwhen creating each piece of stream data. Also, it may be prescribed thatthis flag is “prohibited from being changed during between PIs in whichvideo images are seamlessly connected via CC=5, 6”.

(1-B-2) Offset metadata may be stored in each VAU (i.e., each frame orfield) instead of only being stored in the top VAU in each videosequence (i.e., each GOP). Alternatively, offset metadata may be set atarbitrary intervals, such as three frames or greater, for each content.In this case, it is preferable that offset metadata always be stored inthe top VAU in each video sequence and that the interval between theoffset metadata and the immediately prior offset metadata be restrictedto equal to or greater than three frames. Accordingly, the playbackdevice can reliably perform processing to change offset information inparallel with interrupt playback.

(1-B-3) Instead of being stored in the video stream, offset metadata maybe multiplexed in a main TS or a sub-TS as independent stream data. Inthis case, a unique PID is allocated to the offset metadata. The systemtarget decoder refers to this PID to separate the offset metadata fromother stream data. Alternatively, the offset metadata may first bepreloaded into a dedicated buffer and later undergo playback processing,like the text subtitle stream. In this case, the offset metadata isstored at constant frame intervals. Accordingly, a PTS is not necessaryfor the offset metadata, thus reducing the data amount of the PESheader. This reduces the capacity of the buffer for preloading.

(1-B-4) Instead of being stored in the supplementary data of a VAU,offset metadata may be embedded in the video stream with use of a videowatermark. Furthermore, the offset metadata may be embedded in the audiostream with use of an audio watermark.

(1-B-5) In the sub-TS according to Embodiment 1 of the presentinvention, as shown in FIG. 15, the TS packets 1530 and 1550 located atthe ends of the first group 1521 and the second group 1522,respectively, include AD fields 1532 and 1552 in general. With thisstructure, the three groups 1521-1523 are separated from each other.Alternatively, in the VAU #1 1500 in the dependent-view video stream,the size of the padding data 1506 may be adjusted so that the threegroups 1521-1523 are separated from each other.

(1-B-6) TS packets containing offset metadata may be selected in asystem target decoder depending on PIDs, instead of TS priority. FIG. 50is a schematic diagram showing a PES packet 5010 containing VAU #1 5000in the dependent-view video stream and a sequence of TS packets 5020generated from the PES packet 5010. The VAU #1 5000 is located at thetop of the video sequence, and accordingly includes supplementary data5004 consisting only of offset metadata 5009. The PES payload 5012 ofthe PES packet 5010 contains the VAU #1 5000, and the PES header 5011thereof includes DTS and PTS assigned to compressed picture data 5005 inthe VAU #1 5000. The PES packet 5010 is stored in the TS packet sequence5020 in order from the top. With this arrangement, the TS packetsequence 5020 is divided into three groups 5021, 5022, and 5023 in orderfrom the top. The first group 5021 includes the PES header 5011, sub-AUidentification code 5001, sub-sequence header 5002, and picture header5003. The second group 5022 includes the supplementary data 5004consisting only of the offset metadata 5009. The third group 5013includes the compressed picture data 5005, padding data 5006, sequenceend code 5007, and stream end code 5008. Hatched areas in FIG. 50 showthe supplementary data 5004 consisting only of the offset metadata 5009,and dotted areas show data 5011, 5001-5003 arranged before thesupplementary data in the PES packet 5010. Like the TS packets in thesequence 1520 shown in FIG. 15, the TS packets 5030 and 5050 located atthe ends of the first group 5021 and the second group 5022,respectively, include AD fields 5032 and 5052 in general. With thisstructure, the three groups 5021-5023 are separated from each other. TheTS headers 5031 and 5061 of the TS packets 5030 and 5060 belonging tothe first group 5021 and the third group 5023 each indicate PID=0x1012.Here, the TS headers 5031 of the TS packets 5030 belonging to the firstgroup 5021 may indicate PID=0x1022. On the other hand, the TS headers5041 and 5051 of the TS packets 5040 and 5050 belonging to the secondgroup 5022 each indicate PID=0x1022. The hexadecimal value “0x1022” maybe replaced to any other value except the hexadecimal values assigned tothe other elementary streams. In this way, TS packets belonging to thesecond group 5022 have a different PID from TS packets belonging to thethird group 5023. Accordingly, the system target decoder can easilyselect TS packets belonging to the second group by using PIDs.

The system target decoder extracts offset metadata from the TS packetsequence 5020 shown in FIG. 50 as follows. FIG. 51 is a functional blockdiagram showing a system of processing video streams in the systemtarget decoder 5125. The system target decoder 5125 shown in FIG. 51, incontrast to that 4225 shown in FIG. 45, does not include the TS priorityfilter 4551. Other components thereof are similar to corresponding ones.In FIG. 51, components similar to ones shown in FIG. 45 are marked withthe same reference numbers. Furthermore, details of the similarcomponents can be found in the description on FIG. 45.

The second PID filter 4514 transfers TS packets with PID=0x1012 to TB24508 in the primary video decoder 4515, and transfers TS packets withPID=0x1022 to the offset metadata processing unit 4552. Here, TS packetswith PID=0x1022 may be transferred to TB2 4508 in parallel. In this way,TS packets containing offset metadata are transferred to the offsetmetadata processing unit 4552.

Note that data different from either of TS priority and PID may be usedto select TS packets containing offset metadata from a sub-TS. If thedata, like TS priority and PID, allows a different value to be set foreach TS packet, the data can be used to select the above-described TSpackets. This would be obvious for a person skilled in the art from theabove-described embodiment.

(1-B-7) The offset metadata 1110 according to Embodiment 1 of thepresent invention, as shown in FIG. 11, provides each frame with offsetinformation. Alternatively, when the video stream represents a frame inthe interlace method (e.g. 601), the display unit is not a frame, but afield. In that case, the offset metadata may provide each field with theoffset information or provide the pair of fields constituting each framewith the offset information.

(1-B-8) In the offset metadata according to Embodiment 1 of the presentinvention, each offset sequence defines an offset value for each frame.Alternatively, each offset sequence may define a function thatrepresents a change over time in the offset value for each presentationtime, i.e. a completion function. In this case, the 3D playback deviceuses the completion function at each presentation time to calculate theoffset value for each frame included in that presentation time.

FIG. 52A is a schematic diagram showing a data structure of offsetmetadata 5200 that uses a completion function. As shown in FIG. 52A, theoffset metadata 5200 includes a correspondence table between offsetsequence IDs 5210 and offset sequences 5220. An offset sequence 5220includes a starting offset value (offset_start) 5221, an ending offsetvalue (offset_end) 5222, offset function ID (offset_func_id) 5223, andoffset duration (offset_duration) 5224. When the offset metadata 5200 isstored in a video sequence in the dependent-view video stream, thestarting offset value 5221 indicates the offset value for the firstframe represented by the video sequence. The ending offset value 5222indicates the offset value for the first frame represented by the nextvideo sequence. The offset function ID 5223 defines the type ofcompletion function. The type of completion function represents theshape of the changes in the offset value during the presentation time ofthe video sequence. The offset duration 5224 indicates the length of thepresentation time of the video sequence.

FIG. 52B is a graph showing the types of elements in the completionfunction. As shown in FIG. 52B, the x-axis represents the presentationtime, and the y-axis represents the offset value. In this context, thesign of the offset value is determined by the depth of the graphicsimage, i.e. by whether the 3D graphics image is further back or closerthan the screen. Three types of elements in a completion function areprovided: a linear shape LNR, a convex shape CVX, and a concave shapeCCV. The linear shape LNR is defined by a linear function y=ax+b,whereas the convex shape CVX and concave shape CCV are defined by asecond degree curve y=ax²+bx+c, a third degree curve y=ax³+bx²+cx+d, ora gamma curve y=a(x+b)^(1/r)+c. In this context, the constants a, b, c,and d are parameters determined by the xy coordinates of each edge A, Bof each element, i.e. by a pair of presentation time and the offsetvalue at that point. On the other hand, the constant r is separatelydefined and is stored in each offset sequence. The types of completionfunctions are defined by one of these elements LNR, CVX, and CCV or by acombination thereof

FIG. 52C is a graph showing offset values calculated by a 3D playbackdevice from offset sequence IDs=0, 1, 2 shown in FIG. 52A. As shown inFIG. 52C, the horizontal axis of the graph represents the time elapsedsince the first frame in each video sequence was displayed; in the videosequence, an offset sequence is stored. The black circles A0, B0, A1,B1, A2, and B2 indicate coordinates defined by either the startingoffset value 5221 or ending offset value 5222 and the offset duration5224. The lines GR0, GR1, and GR2 that respectively connect the pairs ofblack circles A0+B0, A1+B1, and A2+B2 represent completion functionsthat are each determined by the type of completion function specified inthe offset function ID 5223 and by the coordinate values of the blackcircles A0+B0, A1+B1, and A2+B2 at the edges of the lines. In the offsetsequence with offset sequence ID=0, the offset function ID 5223indicates “linear”, and thus the black circles A0 and B0 at either edgeare connected by a line #0 GR0 with a linear shape LNR. In the offsetsequence with offset sequence ID=1, the offset function ID 5223indicates “curve #1”, and thus the black circles A1 and B1 at eitheredge are connected by a line #1 GR1 with a convex shape CVX. In theoffset sequence with offset sequence ID=2, the offset function ID 5223indicates “curve #2”, and thus the black circles A2 and B2 at eitheredge are connected by a line #2 GR2 that is formed by a combination of aconvex shape CVX and a concave shape CCV. The white circles representpairs of a presentation time for a frame and an offset value for theframe as calculated by the 3D playback device using the completionfunction indicated by each of the lines GR0, GR1, and GR2. As is clearfrom these lines GR0, GR1, and GR2, the mere combination of the startingoffset value 5221, ending offset value 5222, offset function ID 5223,and offset duration 5224 can represent a variety of changes in offsetvalue, i.e. in the depth of 3D graphics images. Accordingly, the size ofthe overall offset metadata can be reduced without a loss in the abilityto express 3D graphics images.

(1-C) In the AV stream file for 3D video images, data regarding theplayback format of 3D video images may be added to the PMT 1810 shown inFIG. 18. In this case, the PMT 1810 includes 3D descriptors in additionto the PMT header 1801, descriptors 1802, and pieces of streaminformation 1803. The 3D descriptors are information on the playbackformat of 3D video images, are shared by the entire AV stream file, andparticularly include 3D format information. The 3D format informationindicates the playback format, such as L/R mode or depth mode, of the 3Dvideo images in the AV stream file. Each piece of stream information1803 includes 3D stream descriptors in addition to a stream type 1831, aPID 1832, and stream descriptors 1833. The 3D stream descriptorsindicate information on the playback format of 3D video images for eachelementary stream included in the AV stream file. In particular, the 3Dstream descriptors of the video stream include a 3D display type. The 3Ddisplay type indicates whether the video images indicated by the videostream are a left view or a right view when the video images aredisplayed in L/R mode. The 3D display type also indicates whether thevideo images indicated by the video stream are 2D video images or depthmaps when the video images are displayed in depth mode. When a PMT thusincludes information regarding the playback format of 3D video images,the playback system of these video images can acquire such informationsimply from the AV stream file. This sort of data structure is thereforeuseful when distributing 3D video content via a broadcast.

(1-D) Clip Information File

The dependent-view clip information file may include, among streamattribute information 2220 such as in FIG. 22, a predetermined flag inthe video stream attribute information allocated to PID=0x1012, 0x1013of the dependent-view video stream. When turned on, this flag indicatesthat the dependent-view video stream refers to the base-view videostream. Furthermore, the video stream attribute information may includeinformation regarding the base-view video stream to which thedependent-view video stream refers. This information can be used toconfirm the correspondence between video streams when verifying, via apredetermined tool, whether the 3D video content has been created inaccordance with a prescribed format.

According to Embodiment 1 of the present invention, the size ofbase-view extents and dependent-view extents can be calculated from theextent start points 2242 and 2420 included in the clip information file.Alternatively, a list of the size of each extent may be stored in, forexample, the clip information file as part of the meta data.

(1-E) Playlist File

(1-E-1) The 3D playlist file 222 shown in FIG. 31 includes one sub-path.Alternatively, the 3D playlist file may include a plurality ofsub-paths. For example, if the sub-path type of one sub-path is “3DL/R”, then the sub-path type of the other sub-path may be “3D depth”. Byswitching between these two types of sub-paths when playing back 3Dvideo images in accordance with the 3D playlist file, the playbackdevice 102 can easily switch between L/R mode and depth mode. Inparticular, such switching can be performed more rapidly than switchingthe 3D playlist file itself.

A plurality of dependent-view video streams may represent the same 3Dvideo images in combination with a shared base-view video stream.However, the parallax between the left view and right view for the samescene differs between the dependent-view video streams. Thesedependent-view video streams may be multiplexed into one sub-TS, orseparated into different sub-TSs. In this case, the 3D playlist fileincludes a plurality of sub-paths. Each sub-path refers to a differentdependent-view video stream. By switching between sub-paths when playingback 3D video images in accordance with the 3D playlist file, theplayback device 102 can easily change the sense of depth of the 3D videoimages. In particular, such processing can be performed more rapidlythan switching the 3D playlist file itself.

FIG. 53 is a schematic diagram showing (i) a data structure of a 3Dplaylist file 5300 that includes a plurality of sub-paths and (ii) adata structure of a file 2D 5310 and two files DEP 5321 and 5322 thatare referred to by the 3D playlist file 5300. The file 2D 5310 includesa base-view video stream with a PID=0x1011. The file DEP #1 5321includes a dependent-view video stream #1 with a PID=0x1012. The fileDEP #2 5322 includes a dependent-view video stream #2 with a PID=0x1013.In combination with the base-view video stream in the file 2D 5310, thedependent-view video streams #1 and #2 separately represent the same 3Dvideo images. However, the parallax between the left view and right viewfor the same scene differs between the dependent-view video streams #1and #2. Furthermore, offset sequences with the same offset sequence IDdefine different offset values for the same frame number.

The 3D playlist file 5300 includes a main path 5330 and two sub-paths5331 and 5332. The PI #1 of the main path 5330 refers to file 2D 5310,in particular to the base-view video stream. The SUB_PI #1 of each ofthe sub-paths 5331 and 5332 shares the same playback time as the PI #1in the main path 5330. The SUB_PI #1 of the sub-path #1 5331 refers tothe file DEP #1 5321, in particular to the dependent-view video stream#1. The SUB_PI #1 of the sub-path #2 5332 refers to the file DEP #25322, in particular to the dependent-view video stream #2. This is alsotrue in the PI #2 of the main path 5330 and the SUB_PI #2 of each of thesub-paths 5331 and 5332.

During 3D playlist playback processing of the 3D playlist file 5300, theplayback device 102 first has a user or an application program selectthe sub-path for playback. Alternatively, the playback device 102 mayselect the sub-path for playback according to the screen size of thedisplay device 103, or may select the sub-path by referring to theinterpupillary distance of the viewer. By selecting the sub-path in thisway, the parallax between the left-view and right-view video planes caneasily be changed. Furthermore, since offset information changes causedby switching of the dependent-view video stream, the offsets of thegraphics planes played back from the PG stream or IG stream included inthe file 2D 5310 change. This makes it easy to change the sense of depthof the 3D video images.

(1-E-2) In the 3D playlist file shown in FIG. 31, the base-view videostream is registered in the STN table 3205 in the main path 3101, andthe dependent-view video stream is registered in the STN table SS 3130in the extension data 3103. Alternatively, the dependent-view videostream may be registered in the STN table. In that case, the STN tablemay include a flag indicating which of the base view and the dependentview is represented by the registered video stream.

(1-E-3) According to Embodiment 1 of the present invention, 2D playlistfiles and 3D playlist files are stored separately in the BD-ROM disc101. Alternatively, in a similar manner to the extension data 3103, thesub-path 3102 shown in FIG. 31 may be recorded in an area that isreferenced only by the playback device 102 in the 3D playback mode. Inthat case, the 3D playlist files as they are can be used as the 2Dplaylist files since there is no risk that the sub-path 3102 causes theplayback device 102 in the 2D playback mode to malfunction. As a result,the authoring of the BD-ROM disc is simplified.

(1-E-4) The reference offset IDs and offset adjustment values for the PGstream, IG stream, and text subtitle stream may be stored in the STNtable SS 3130 instead of in the STN table 3205. Alternatively, thisinformation may be stored in the stream attribute information 2220 inthe clip information file. Furthermore, the reference offset ID may bestored in the subtitle entry for each PG stream and text subtitle streamor may be stored in each page of the IG stream.

(1-E-5) When reference offset IDs are set in the 3D playlist file, thefollowing constraint conditions may be prescribed for seamlessconnection between PIs. For example, when CC=5 is set in the PI #2 ofthe main path 5330 shown in FIG. 53, video images in the playbacksections defined by PI #1 and PI #2 need to be connected seamlessly. Inthis case, in PI #1 and PI #2, changes are prohibited to both the valuesof the reference offset IDs and the number of offset sequences includedin the dependent-view video stream, i.e. the number of entries.Furthermore, changes to both the offset adjustment values and the numberof entries thereof may be prohibited. With these constraint conditions,the playback device 102 can skip updating of the SPRM(27) when changingthe current PI from PI #1 to PI #2. Since the processing load forseamless connection is thus reduced, the reliability of this processingcan be further improved. As a result, the quality of 3D video images canbe improved.

(1-E-6) In the STN table, two or more offset adjustment values may beset for one piece of stream data. FIG. 54 is a schematic diagram showingsuch an STN table 5400. As shown in FIG. 54, the STN table 5400associates an STN 5401 with a stream entry 5410 of PG stream 1 and apiece of stream attribute information 5403. The stream attributeinformation 5403 includes three types of offset adjustment values #1-#35411-5413 along with a reference offset ID 5410. These offset adjustmentvalues are used to change an offset depending on the screen size of adisplay device; the offset is to be provided to a graphics planegenerated from PG stream 1.

Assume that the correspondence between the types of offset adjustmentvalues and screen sizes is specified in advance. Specifically, offsetadjustment value #1 5411, offset adjustment value #2 5412, and offsetadjustment value #3 5413 are respectively used when the screen sizefalls within the range of 0-33 inches, 34-66 inches, and 67 inches orgreater. The offset adjustment values 5411-5413 are set to satisfy thefollowing condition: parallaxes between left-view and right-viewgraphics images produced by providing an offset to a graphics plane havethe maximum value equal to or less than an interpupillary distance of ageneral viewer (in the case of a child, 5 cm or less). As long as thiscondition is satisfied, the parallax will not exceed the viewer'sinterpupillary distance. This can reduce the viewer's risk of sufferingvisually-induced motion sickness and eye strain.

Each time the change of current PI causes the change of the total numberof offset adjustment values allocated to a piece of stream data, theplayback control unit 4235 of the playback device 102 selects offsetadjustment values to be used actually depending on the screen size ofthe display device 103. Specifically, the playback control unit 4235first acquires the screen size of the display device 103, if necessary,by performing the HDMI authentication. The playback control unit 4235next selects one of offset adjustment value #1-#3, 4801-4803, dependingon which range the screen size of the display device 103 falls within;0-33 inches, 34-66 inches, 67 inches or greater. The playback controlunit 4235 stores information representing the type of the selected valueas a player variable into the player variable storage unit 4236. Thus,until the selection of offset adjustment values are performed again, theplayback control unit 4235 selects offset adjustment values of the typeindicated by the player variable from each STN table, and then updatesthe value of the SPRM(28) to the selected value.

(1-F) The index file 211 shown in FIG. 35 includes the 3D existence flag3520 and 2D/3D preference flag 3530 shared by all the titles.Alternatively, the index file may specify a different 3D existence flagor 2D/3D preference flag for each title.

(1-G) SPRM(27), SPRM(28)

(1-G-1) The program execution unit 4234 may set the values of theSPRM(27) 4751 and the SPRM(28) 4752 in accordance with a movie object orBD-J object. In other words, the playback device 102 may cause anapplication program to set the reference offset ID and offset adjustmentvalue. Furthermore, such an application program may be limited to anobject associated with the item “first play” 3501 in the index table3510.

(1-G-2) The playback control unit 4235 may have the viewer adjust theoffset to be provided to the graphics plane. Specifically, when theviewer operates the remote control 105 or the front panel of theplayback device 102 and requests to set the offset adjustment value,first the user event processing unit 4233 receives the request andnotifies the playback control unit 4235 of the request. Next, inresponse to the request, the playback control unit 4235 displays anoperation screen for adjusting the offset on the display device 103.Here, an OSD of the playback device 102 is used for displaying thisoperation screen. The playback control unit 4235 further has the viewerselect a graphics plane for adjustment and increase/decrease of theoffset value, through operation of the remote control 105 or the like.The playback control unit 4235 then updates SPRM(28) to add or subtracta predetermined value to/from the offset adjustment value correspondingto the selected graphics plane. Preferably, during the adjustmentprocessing, the playback control unit 4235 causes the playback unit 4202to continue playback processing of the graphics plane. Here, theplayback unit 4202 makes the operation screen or the graphicsimage—whichever is displayed closer to the viewer—semi-transparent, ordisplays the operation screen closer than the graphics image. This makesthe graphics image visible even when the operation screen is beingdisplayed, and thus the viewer can immediately confirm the effect ofincreasing or decreasing the offset value in the same way as whenadjusting the brightness or color of the screen.

(1-G-3) For offset control, each of the cropping units 4731-4734 shownin FIG. 47 uses the offset sequence specified by the reference offsetIDs indicated by the SPRM(27). Conversely, for offset control, eachcropping unit 4731-4734 may be made not to use the offset sequencespecified by each offset sequence ID indicated by a predetermined SPRM.In other words, the SPRM may indicate the offset sequence IDs(PG_ref_offset_id_mask, IG_ref_offset_id_mask, SV_ref_offset_id_mask,IM_ref_offset_id_mask) that are to be masked during offset control. Inthis case, each of the cropping units 4731-4734 may select the ID of theoffset sequence that includes the largest offset value from among theoffset sequences that are received from the system target decoder 4225and are allocated to the offset sequence IDs not masked in the offsetinformation 4707. In this way, the depth of the graphics imagesrepresented by the secondary video plane, PG plane, IG plane, and imageplane can easily be aligned. This allows for an increase in the degreeof freedom when creating each piece of stream data.

(1-H) When displaying a menu unique to the playback device 102 as anOSD, the playback device 102 may perform offset control on the graphicsplane representing the 2D video images in the menu, i.e. on the OSDplane. In this case, the playback device 102 may select, within theoffset information transmitted by the system target decoder 4225 at thepresentation time of the menu, the offset information that has an offsetdirection that is closer to the viewer than the screen and that has thelargest offset value. The menu can thus be displayed closer than any 3Dgraphics image, such as subtitles or the like, played back from the 3Dvideo content.

Alternatively, the playback device 102 may pre-store offset informationfor the OSD plane. A specific offset sequence ID, such as offset_id=0,is allocated to this offset information. Furthermore, the following twoconditions may be placed on the offset information with an offsetsequence ID=0: (1) The offset direction is closer to the viewer than thescreen, and (2) The offset value is the same as the largest offset valueamong those included in the pieces of offset information that (i) areallocated to offset sequence IDs other than zero, (ii) correspond to thesame frame number, and (iii) have offset directions closer to the screenthan the viewer. With this prescription, the playback device 102 doesnot have to select offset information from among the offset informationtransmitted by the system target decoder 4225, thus simplifying offsetcontrol of the OSD plane. Also, each of the cropping units 4731-4734 mayuse offset information for offset sequence ID=0 as a substitute whenunable to detect reference offset IDs indicated by SPRM(27) among theoffset information 4707 received from the system target decoder 4225.

(1-I) In the 3D playback device, in addition to the setting of parentallevel in SPRM(13), 3D parental level may be set in SPRM(30). The 3Dparental level indicates a predetermined restricted age and is used forparental control of viewing of 3D video titles recorded on the BD-ROMdisc 101. Like the value in SPRM(13), a user of the 3D playback devicesets the value of the SPRM(30) via, for example, an OSD of the 3Dplayback device. The following is an example of how the 3D playbackdevice performs parental control onto each 3D video title. The 3Dplayback device first reads, from the BD-ROM disc 101, the age for whichviewing of a title in the 2D playback mode is permitted and comparesthis age with the value of the SPRM(13). If this age is equal to or lessthan the value of the SPRM(13), the 3D playback device stops playback ofthe title. If this age is greater than the value of the SPRM(13), the 3Dplayback device reads, from the BD-ROM disc 101, the age for whichviewing of a title in the 3D playback mode is permitted and comparesthis age with the value of the SPRM(30). If this age is equal to orgreater than the value of the SPRM(30), the 3D playback device playsback the title in the 3D playback mode. If this age is less than thevalue of the SPRM(30) and equal to or greater than the value of theSPRM(13), the 3D playback device plays back the title in the 2D playbackmode. In this way, the difference in viewer's interpupillary distance bythe age taken into account, it is possible to realize a parental controlso that, for example, “children whose ages are less than a predeterminedvalue can view 3D video images only as 2D video images”. Preferably theparental control is performed when it is judged that “the display devicesupports playback of 3D video images” in the processing of selecting aplaylist file for playback shown in FIG. 36, namely when it is judgedYES in step S3605. Note that a value indicating permission/prohibitionof 3D playback mode may be set in SPRM(30) instead of parental level,and the 3D playback device may judge whether the 3D playback mode isvalid or invalid in accordance with the value.

(1-J) In the 3D playback device, a value indicating “which of 2Dplayback mode and 3D playback mode is to be prioritized” may be set inSPRM(31). A user of the 3D playback device sets the value of theSPRM(31) via, for example, an OSD of the 3D playback device. In stepS3603 in the processing of selecting a playlist file for playback shownin FIG. 36, the 3D playback device refers to the SPRM(31) as well as the2D/3D preference flag. When both the SPRM(31) and 2D/3D preference flagindicate the 2D playback mode, the 3D playback device selects the 2Dplayback mode. When both the SPRM(31) and 2D/3D preference flag indicatethe 3D playback mode, the 3D playback device proceeds to step S3605 andperforms the HDMI authentication, without displaying the playback modeselection screen. As a result, when the display device is supporting the3D video images, the 3D playback device selects the 3D playback mode.When the SPRM(31) and 2D/3D preference flag indicate different playbackmodes, the 3D playback device executes step S3604, i.e. displays theplayback mode selection screen to have the user select a playback mode.Alternatively, the 3D playback device may have the application programselect a playback mode. In this way, even if the 2D/3D preference flagis set in the 3D video content, it is possible to have the user select aplayback mode only when the playback mode indicated by the 2D/3Dpreference flag does not match the playback mode indicated by theSPRM(31) which is the playback mode having been set by the user inadvance.

An application program such as a BD-J object may select a playback modeby referring to the SPRM(31). Furthermore, the application program maydetermine the initial state of the menu to be displayed on the selectionscreen depending on the value of the SPRM(31), when causing a user toselect a playback mode at step S3604 shown in FIG. 36. For example, whenthe value of the SPRM(31) indicates that the 2D playback mode has a highpriority, the menu is displayed in the state in which a cursor ispositioned on a button for selecting the 2D playback mode; when thevalue of the SPRM(31) indicates that the 3D playback mode has a highpriority, the menu is displayed in the state in which the cursor ispositioned on a button for selecting the 3D playback mode.Alternatively, when the 3D playback device has a function to manage theaccounts of a plurality of users such as a father, a mother, and achild, the 3D playback device may set a value to the SPRM(31) dependingon the account of a user who is logged in at the current time.

The value of the SPRM(31) may indicate “which of 2D playback mode and 3Dplayback mode is to be always set”, in addition to “which of 2D playbackmode and 3D playback mode is to be prioritized”. When the value of theSPRM(31) indicates “2D playback mode is to be always set”, the 3Dplayback device always selects the 2D playback mode irrespectively ofthe value of the 2D/3D preference flag. In that case, the value of theSPRM(25) is set to indicate the 2D playback mode. When the value of theSPRM(31) indicates “3D playback mode is to be always set”, the 3Dplayback device performs the HDMI authentication without displaying theplayback mode selection screen irrespectively of the value of the 2D/3Dpreference flag. In that case, the value of the SPRM(25) is set toindicate the 3D playback mode (L/R mode or depth mode). In this way,even if the 2D/3D preference flag is set in the 3D video content, it ispossible to allow the playback mode having been set by the user inadvance to be always prioritized.

(1-K) The playback device 102 may have the user register aninterpupillary distance as a reserved SPRM, for example SPRM(32). Inthis case, the playback device 102 can adjust the offset adjustmentvalue so that the maximum value of the parallax between the left-viewand right-view graphics images does not exceed the value registered inthe SPRM(32). Specifically, it suffices for the playback device 102 toperform the following calculations for each offset value output by thesystem target decoder. The playback device 102 first seeks the ratio ofthe value of the SPRM(32) to the width (horizontal length) of the screenof the display device 103 and further seeks the product of this ratioand the number of horizontal pixels of the display device 103. Thisproduct represents two times the upper limit of the offset that can beprovided to the graphics plane via offset control. Next, the playbackdevice 102 compares this product with the double of each offset value.If the double of any offset value is equal to or greater than thisproduct, the playback device 102 identifies the ID of the offsetsequence that includes the offset value and reduces the offsetadjustment value for the graphics plane indicated by that ID. The amountof the reduction is set to at least half the difference between thedouble of the offset value and the above product. The maximum value ofthe parallax between a left-view and a right-view graphics image thusdoes not exceed the viewer's interpupillary distance. This can reducethe viewer's risk of suffering visually-induced motion sickness and eyestrain.

(1-L) Output Offset Adjustment

(1-L-1) Since the alternate-frame sequencing method represents aparallax between left and the right views by the number of pixels in thehorizontal direction, the actual size of the parallax depends on thescreen size of a display device, namely, the size of a pixel. On theother hand, the sense of depth of 3D video images depends on the actualsize of the parallax. Accordingly, in order to avoid the sense of depthof the 3D video images on a screen of any size from impairing powerfulimpression of the 3D video images and fatiguing viewer's eyesexcessively, the parallax between the left and right views needs to beadjusted to be appropriate to the screen size. As one method of theadjustment, the 3D playback device further provides an offset to thefinal frame data combined by the plane adders. The offset is provided ina manner similar to that in which an offset is provided to a graphicsplane in the 1 plane+offset mode. The offset control that is furtherapplied to the final frame data is referred to as “output offsetadjustment”.

FIGS. 55A-55C are schematic diagrams showing parallaxes PRA, PRB, andPRC between left and right views displayed on a 32-inch screen SCA,50-inch screen SCB, and 100-inch screen SCC, respectively. The imagesLA1, LA2, LB, LC1, and LC2 drawn on the figures by solid lines representleft views, and the images RA1, RA2, RB, RC1, and RC2 drawn by dashedlines represent right views. Here, assume that video content specifiesparallaxes between left and right views to produce an optimal sense ofdepth when 3D video images are displayed on a 50-inch screen. As shownin FIG. 55B, the parallax between the left view LB and the right view RBis equal to an optimal value PRB when a 3D video image represented bythe video content is displayed on a 50-inch screen SCB.

The parallax DA between the left view LA1 and the right view RA1 drawnby thin lines in FIG. 55A is equal in number of horizontal pixels to theparallax DB between the left view LB and the right view RB drawn in FIG.55B. On the other hand, a pixel on the 32-inch screen SCA is smallerthan a pixel on the 50-inch screen SCB. Accordingly, 3D video imagesgenerated by the parallax DA between the left view LA1 and the rightview RA1 drawn by the thin lines produce a sense of depth weaker thanthe optimal one, in general. In that case, the output offset adjustmentincreases the parallax between the left view and the right view by twicea predetermined adjustment value CRA. The parallax PRA between the leftview LA2 and the right view RA2 drawn by thick lines in FIG. 55Aindicates a parallax after the output offset adjustment. In this way,when a parallax between left and right views is increased, a sense ofdepth of 3D video images is enhanced. This prevents the 3D video imagesfrom losing powerful impression.

The parallax DC between the left view LC1 and the right view RC1 drawnby thin lines in FIG. 55C is equal in number of horizontal pixels to theparallax PRB between the left view LB and the right view RB drawn inFIG. 55B. On the other hand, a pixel of the 100-inch screen SCC islarger than a pixel of the 50-inch screen SCB. Accordingly, 3D videoimages generated by the parallax DC between the left view LC1 and theright view RC1 drawn by the thin lines produce a sense of depth strongerthan the optimal one, in general. In that case, the output offsetadjustment decreases the parallax between the left view and the rightview by twice a predetermined adjustment value CRC. The parallax PRCbetween the left view LC2 and the right view RC2 drawn by thick lines inFIG. 55C indicates the parallax after the output offset adjustment. Inthis way, when a parallax between left and right views is decreased, asense of depth of 3D video images is suppressed. This prevents a viewerfrom suffering eye fatigue.

The adjustment values CRA and CRC shown in FIGS. 55A and 55C arereferred to as “output offset adjustment values”. Video content includesa correspondence table between screen sizes and output offset adjustmentvalues stored in an index file, playlist file, or clip information file.FIG. 56A is a schematic diagram showing the correspondence table. Asshown in FIG. 56A, an output offset adjustment value is set for eachrange of screen size with a width of 10 inches. The magnitude of eachoutput offset adjustment value indicates the number of horizontalpixels, and the sign thereof indicates increase/decrease of the parallaxbetween left and right views. Note that the range of screen size may hasa width other than 10 inches. The correspondence table is set accordingto standards or by a user. Furthermore, two or more types ofcorrespondence tables may be recorded in a 3D playback device inadvance, and video content may specify the identifier of a type of thecorrespondence tables; the type is to be used during playback of thevideo content.

(1-L-2) A 3D playback device may use a predetermined function to selectan output offset adjustment value, instead of the above-describedcorrespondence table. FIG. 56B is a graph representing the function. Thehorizontal axis of this graph indicates the screen size in inch, and thevertical axis thereof indicates the output offset adjustment valuerepresented by the number of pixels with a sign. The 3D playback deviceuses the function represented by the graph to calculate an output offsetadjustment value from the screen size of a display device. As the graphshows, the output offset adjustment value is a greater positive valuewhen the screen size is smaller than 50 inches, and a greater negativevalue when the screen size is greater than 50 inches. Note that theoutput offset adjustment value is maintained at a substantially constantpositive value when the screen size is 32 inches or less, and the outputoffset adjustment value is maintained at a substantially constantnegative value when the screen size is 103 inches or more.

(1-L-3) Video content may include an optimal value of screen size thatis assumed at the time of authoring (assumed_TV_size_when_authoring),and a 3D playback device may determine an output offset adjustment valuebased on the optimal value. For example, when the screen size of adisplay device exceeds the optimal value, the 3D playback device firstreduces the frame size of 3D video images to the optimal value. The 3Dplayback device next superimposes a black border on the edges of eachframe, and causes the entirity of the frame and black border to be equalin size the screen of the display device. The 3D playback device furtheradjusts the output offset adjustment value so that the parallax betweenthe left and right views displayed inside the black border is equal inmagnitude to a parallax between the left and right views if they aredisplayed on the entirity of a screen with a size equal to the optimalvalue. This enables a sense of depth of the 3D video images to bemaintained equal to that assumed at the time of authoring.

(1-L-4) FIG. 57 is a block diagram showing the components of a 3Dplayback device required for the output offset adjustment. The 3Dplayback device 5700 shown in FIG. 57, in contrast to that 4200 shown inFIG. 42, includes an output offset adjustment value application unit5701. Other components thereof are similar to corresponding ones. InFIG. 57, the components similar to ones shown in FIG. 42 are marked withthe same reference numbers. Furthermore, details of the similarcomponents can be found in the description on FIG. 42.

The player variable storage unit 4236 stores output offset adjustmentvalues in the SPRM(36). The output offset adjustment values are based ona screen size obtained by the HDMI communication unit 4237 from thedisplay device 103 through HDMI authentication, and determined by theplayback control unit 4235 using the correspondence table or thefunction shown in FIG. 56A or 56B. Alternatively, an application programsuch as a BD-J object may automatically set the value of the SPRM(35) orcause a user to set the value by using a GUI.

The output offset adjustment value application unit 5701 uses an outputoffset adjustment value indicated by the SPRM(35) to provide an offsetto each of left-view and right-view frame data combined by the planeadders 4226. The output offset adjustment of the frame data by theoutput offset adjustment value application unit 5701 is similar to theoffset control over the PG plane data GP by the second cropping unit4732 shown in FIG. 49.

(1-L-5) The screen size of the display device 103 may be stored in theSPRM(35) instead of output offset adjustment values. In that case, theoutput offset adjustment value application unit 5701 retrieves an outputoffset adjustment value associated with the screen size indicated by theSPRM(35) from the correspondence table shown in FIG. 56A.

(1-L-6) The player variable storage unit 4236 may further store anoutput offset adjustment value alpha into the SPRM(36). The outputoffset adjustment value alpha represents a positive numeric value. Theoutput offset adjustment value application unit 5701 uses the product ofan output offset adjustment value indicated by the SPRM(35) and anoutput offset adjustment value alpha indicated by the SPRM(36) as anactual output offset adjustment value. This enables depths of 3D videoimages adjusted by the output offset adjustment to depend on not onlyscreen sizes but also the ages of viewers. For example, when the viewersinclude a child with a smaller interpupillary distance than an adult,the output offset adjustment for small screen size sets the outputoffset adjustment value alpha to a value smaller than “1,” and theoutput offset adjustment for large screen size sets it to a valuegreater than “1.” This weakens a sense of depth of 3D video imagesregardless of the screen size.

The program execution unit 4234 or the playback control unit 4235 mayuse a GUI or an OSD to cause a user to set an output offset adjustmentvalue alpha. In that case, acceptable levels of the output offsetadjustment value alpha may be represented by, for example, the followingthree levels: “the sense of depth of 3D video images is strong”,“normal”, and “weak”. Alternatively, an output offset adjustment valuealpha may be stored in supplementary data in a video stream, adescriptor in a PMT, or a PI in a playlist file included in videocontent. With this structure, the output offset adjustment value alphacan vary with scenes represented by the video stream. In particular, theoutput offset adjustment value alpha can be set to be reduced in a sceneproducing a strong sense of depth.

(1-L-7) Output offset adjustment values may be changed depending on adistance between a viewer and a screen, instead of the screen size.Furthermore, output offset adjustment values alpha may depend on thedistance. In that case, for example, a distance sensor is mounted on theshutter glasses 104 shown in FIG. 1 and used to measure a distancebetween the display device 103 and the screen 131. The distance istransmitted from the shutter glasses 104 to the display device 103 atany time, and further transmitted from the display device 103 to theplayback device 102 via the HDMI cable 122. The playback device 102 usesthe distance to select an output offset adjustment value or an outputoffset adjustment value alpha.

(1-L-8) When the display device 103 is a projector, images are enlargedby lenses and projected onto a screen. Accordingly, the size of adisplay area on the screen changes depending on the distance between theprojector and the screen. In that case, the projector determines thesize of the display area by, for example, either of the following twomethods. The first method first measures the distance between theprojector and the screen, and then calculates the size of the displayarea based on the relationship between the distance and thecharacteristics of the optical system of the projector, especially thespread angle of projection light. Here, a distance sensor mounted on theprojector is used to measure the distance. For example, the distancesensor first emits infrared laser light and the like to the screen, andthen detects reflection light from the screen. At the same time, thedistance sensor also measures the length of the time elapsing from theemission of the laser light until the detection of the reflection light.The distance sensor then calculates the distance between the projectorand the screen from the elapsed time. The second method causes theprojector to operate as follows: the projector first projects areference graphic object such as a line segment onto the screen, andnext uses an OSD or the like to urge a viewer to measure and enter thesize of the reference graphic object on the screen. The projector thencalculates the size of the display area from the size of the referencegraphic object entered by the viewer.

(1-M) In some video content, such as content for displaying song lyricsduring karaoke, graphics image of subtitles or the like are repeatedlydisplayed as still images, and only the graphics images are frequentlyupdated. When such content is formed into 3D video content, the VAU inwhich the offset metadata is placed further includes a sequence endcode. When the playback device 102 decodes this VAU, it stores theoffset information obtained from the offset metadata and does not changethe offset information until a VAU that includes new offset metadata isdecoded.

FIG. 58A is a schematic diagram showing a data structure of adependent-view video stream 5800 representing only still images. EachVAU in the dependent-view video stream 5800 represents one still image.In this case, a sequence end code 5803, 5804 is placed at the end ofeach VAU. Meanwhile, offset metadata 5811, 5812 is placed in thesupplementary data 5801, 5802 of each VAU. The offset metadata 5811 inVAU #1 includes an offset sequence [0] with an offset sequence ID=0.This offset sequence [0] includes only offset information on frame #1.Similarly, in the offset metadata 5812 of VAU #2, the offset sequence[0] includes only offset information on frame #1.

It is assumed here that the 3D playlist file specifies the following twoitems: (1) The still images represented by the VAUs in thedependent-view video stream 5800 switch at 10 second intervals, and (2)Graphics images represented by the graphics stream are overlapped oneach still image. FIG. 58B is a schematic diagram showing a left-viewvideo plane sequence 5821, a right-view video plane sequence 5822, and agraphics plane sequence 5830 that are played back in accordance withsuch a 3D playlist file. In FIG. 58B, the video planes at the time whenthe still image is switched are shown with hatching. In the left-viewvideo plane sequence 5821, the still image indicated by the first videoplane 5841 is repeatedly played back for the first 10 second interval5861, and the still image indicated by the next video plane 5851 isrepeatedly played back for the next 10 second interval 5871. In theright-view video plane sequence 5822, the still image indicated by thefirst video plane 5842 is repeatedly played back for the first 10 secondinterval 5862, and the still image indicated by the next video plane5852 is repeatedly played back for the next 10 second interval 5872.

When the playback device 102 decodes VAU #1 in the dependent-view videostream 5800, it reads offset information for frame #1 from the offsetmetadata 5811. Furthermore, the playback device 102 detects the sequenceend code 5803. At this point, the playback device 102 stores the offsetinformation for frame #1. In this way, during the first 10 secondinterval 5861, the offset provided to the graphics plane sequence 5830is maintained constant in accordance with the stored offset information.In other words, the depth of the graphics images is maintained constant.

Once 10 seconds have passed after decoding of VAU #1, the playbackdevice 102 decodes VAU #2, and reads new offset information for frame #1from the offset metadata 5812. Furthermore, the playback device 102detects the sequence end code 5804. At this point, the playback device102 stores the offset information for frame #1. In this way, during thenext 10 second interval 5871, the offset provided to the graphics planesequence 5830 is changed and maintained constant in accordance with thenewly stored offset information. In other words, the graphics images aremaintained constant at a new depth.

When a VAU includes a sequence end code, the playback device 102 is thuscaused to store existing offset information as is. Accordingly, evenwhen a video stream is composed only of still images, the playbackdevice 102 can reliably maintain offset control for the graphics plane.

(1-N) Compensation of misalignment between left view and right view

There are cases in which a “misalignment” occurs between a left view anda right view. The playback device 102 or the display device 103according to Embodiment 1 of the present invention compensates themisalignment by using the means described below. This prevents the riskthat the misalignment may cause viewers to feel uncomfortable.

The playback device 102 uses the function units shown in FIG. 42 tocompensate the aforementioned misalignment. Alternatively, the displaydevice 103 may perform the compensation processing. FIG. 59 is a blockdiagram of the display device 103 that performs the compensationprocessing. As shown in FIG. 59, the display device 103 includes areceiving unit 5901, a stream processing unit 5902, a signal processingunit 5903, and an output unit 5904. The receiving unit 5901 receivesmultiplexed stream data from various mediums such as a BD-ROM disc,semiconductor memory device, external network, and broadcast wave, aswell as from the playback device 102, and passes the receivedmultiplexed stream data to the stream processing unit 5902. The streamprocessing unit 5902 separates various types of data such as video,audio, and graphics from the multiplexed stream data, and passes thevarious types of data to the signal processing unit 5903. The signalprocessing unit 5903 decodes each of the various types of data, andpasses the results thereof to the output unit 5904. The output unit 5904converts each of the decoded data into a predetermined format, andoutputs the results thereof. The output of the output unit 5904 isvideo/audio itself. Alternatively, the output may be a video/audiosignal in the HDMI format. Except for mechanical parts such as the discdrive, display panel, and speaker, the elements 5901, 5902, 5903, and5904 shown in FIG. 59 are implemented on one or more integratedcircuits.

(1-N-1) Horizontal Misalignment Between Left View and Right View

FIG. 60A is a plan view schematically showing horizontal angles of viewHAL and HAR for a pair of video cameras CML and CMR filming 3D videoimages. As shown in FIG. 60A, the pair of video cameras CML and CMR areplaced side by side in the horizontal direction. The left-video cameraCML films the left view, and the right-video camera CMR films the rightview. The horizontal angles of view HAL and HAR of the video cameras CMLand CMR are the same size but differ in location. This yields a strip ALthat is only included in the horizontal angle of view HAL of theleft-video camera CML and a strip AR that is only included in thehorizontal angle of view HAR of the right-video camera CMR. The objectOBC located in the section common to both horizontal angles of view HALand HAR is captured by both video cameras CML and CMR. However, theobject OBL located in strip AL, which is included only in the horizontalangle of view HAL of the left-video camera CML, is only captured by theleft-video camera CML, and the object OBR located in strip AR, which isincluded only in the horizontal angle of view HAR of the right-videocamera CMR, is only captured by the right-video camera CMR.

FIG. 60B is a schematic diagram showing a left view LV filmed by theleft-video camera CML, and FIG. 60C is a schematic diagram showing aright view RV captured by the right-video camera CMR. As shown in FIGS.60B and 60C, the strip AL, which is included only in the horizontalangle of view HAL of the left-video camera CML, appears as a strip alongthe left edge of the left view LV. However, this strip AL is notincluded in the right view RV. On the other hand, the strip AR, which isincluded only in the horizontal angle of view HAR of the right-videocamera CMR, appears as a strip along the right edge of the right viewRV. However, this strip AR is not included in the left view LV.Accordingly, among the three objects OBL, OBC, and OBR shown in FIG.60A, the object on the right OBR is not included in the left view LV,and the object on the left OBL is not included in the right view RV. Asa result, the object on the left OBL is only visible to the viewer'sleft eye, and the object on the right OBR is only visible to the righteye. The left view LV and right view RV thus run the risk of causing theviewer to feel uncomfortable.

On the BD-ROM disc 101, information indicating the width WDH of theabove strips AL and AR included in each frame of the left view LV andright view RV is stored in the dependent-view video stream. Thisinformation is stored in the supplementary data of the VAU at the top ofeach video sequence. Note however that this supplementary data isdifferent from the supplementary data including the offset metadata 1110shown in FIG. 11. On the other hand, in the playback device 102, thesystem target decoder 4225 reads information showing the width WDH ofthe above strips AL and AR from the dependent-view video stream.Furthermore, the system target decoder 4225 transmits this informationto the parallax video generation unit 4710 in the plane adder 4226 orthe output unit 5904 in the display device 103. When the receiving unit5901 in the display device 103 directly reads a 3D video content from aninformation medium such as a BD-ROM disc, the above-mentionedinformation is read from the dependent-view video stream and transmittedto the output unit 5904 by the signal processing unit 5903 in thedisplay device 103. The parallax video generation unit 4710 or theoutput unit 5904 (hereinafter, referred to as “parallax video generationunit 4710 etc.”) refers to this information to process the left-videoplane and the right-video plane, uniformly painting the strips AL and ARa background color or black. In other words, the pixel data included inthe strips AL and AR is uniformly overwritten with data that representsa background color or black.

FIGS. 60D and 60E are schematic diagrams respectively showing a leftview LV represented by the processed left-video plane and a right viewRV represented by the processed right-video plane. As shown in FIG. 60D,the strip AL, which is included only in the horizontal angle of view HALof the left-video camera CML, is hidden by a black strip BL of widthWDH. On the other hand, as shown in FIG. 60E, the strip AR, which isincluded only in the horizontal angle of view HAR of the right-videocamera CMR, is hidden by a black strip BR of width WDH. As a result,both of the viewer's eyes see only the area shared by the left view LVand the right view RV, which avoids the risk of causing the viewer tofeel uncomfortable.

Furthermore, the parallax video generation unit 4710 etc. may performcropping similar to that shown in FIG. 47 to remove pixel data includedin the outer half of the strips AL and AR respectively located in theleft-video plane and right-video plane. In this case, the parallax videogeneration unit 4710 etc. uniformly paints the remaining half of thestrips AL and AR a background color or black and, in addition, adds abackground-color or black strip of half the width of the strips AL andAR to the opposite side. In this way, both of the viewer's eyes see thearea shared by the left view LV and the right view RV in the center ofthe screen, with background color or black strips at both edges of thescreen. This avoids the risk of causing the viewer to feeluncomfortable.

Alternatively, the parallax video generation unit 4710 etc. may processthe left-video plane and right-video plane as follows. First, viacropping similar to that shown in FIG. 49, the parallax video generationunit 4710 etc. removes the pixel data in the strips AL and AR from eachof the video planes. Next, the parallax video generation unit 4710 etc.resizes each video plane from the pixel data in the remaining area viascaling. The video image shown by the remaining area is thus expanded tofill the entire frame. As a result, both of the viewer's eyes see onlythe area shared by the left view LV and the right view RV, which avoidsthe risk of causing the viewer to feel uncomfortable.

(1-N-2) Vertical Misalignment Between Left View and Right View

FIG. 61A is a plan view schematically showing vertical angles of viewVAL and VAR for a pair of video cameras CML and CMR filming 3D videoimages. As shown in FIG. 61A, the vertical angles of view VAL and VARfor the video cameras CML and CMR are the same size but differ inlocation. This yields a strip AT that is only included in the verticalangle of view VAL of the left-video camera CML and a strip AB that isonly included in the vertical angle of view VAR of the right-videocamera CMR. The object OBJ located in the section common to bothvertical angles of view VAL and VAR is captured by both video camerasCML and CMR. However, objects located in strip AT, which is includedonly in the vertical angle of view VAL of the left-video camera CML, areonly captured by the left-video camera CML, and objects located in stripAB, which is included only in the vertical angle of view VAR of theright-video camera CMR, are only captured by the right-video camera CMR.

FIG. 61B is a schematic diagram showing a left view LV filmed by theleft-video camera CML and a right view RV filmed by the right-videocamera CMR. As shown in FIG. 61B, the strip AT, which is included onlyin the vertical angle of view VAL of the left-video camera CML, appearsas a strip along the top of the left view LV. However, this strip AT isnot included in the right view RV. On the other hand, the strip AB,which is included only in the vertical angle of view VAR of theright-video camera CMR, appears as a strip along the bottom edge of theright view RV. However, this strip AB is not included in the left viewLV. Note that the positions of the strips AT and AB may be reversedbetween the left view LV and right view RV. In this way, when the leftview LV and right view RV differ with regards to inclusion of the stripsAT and AB, the vertical position of the object OBJ shown in FIG. 61Adiffers between the left view LV and the right view RV by the height HGTof the strips AT and AB. As a result, the vertical position of theobject OBJ differs as seen by the viewer's left eye and right eye, whichhas the risk of causing the viewer to feel uncomfortable.

On the BD-ROM disc 101, information indicating the height HGT of theabove strips AT and AB included in each frame of the left view LV andright view RV is stored in the dependent-view video stream. Thisinformation is stored in the supplementary data of the VAU at the top ofeach video sequence. Note however that this supplementary data isdifferent from the supplementary data including the offset metadata 1110shown in FIG. 11. On the other hand, in the playback device 102, thesystem target decoder 4225 reads the information indicating the heightHGT of the above strips AT and AB from the dependent-view video stream.Furthermore, the system target decoder 4225 transmits this informationto the parallax video generation unit 4710 in the plane adder 4226 orthe output unit 5904 in the display device 103. When the receiving unit5901 in the display device 103 directly reads a 3D video content from aninformation medium such as a BD-ROM disc, the above-mentionedinformation is read from the dependent-view video stream and transmittedto the output unit 5904 by the signal processing unit 5903 in thedisplay device 103.

The parallax video generation unit 4710 or the output unit 5904(hereinafter, referred to as “parallax video generation unit 4710 etc.”)refers to the height HGT of the strips AT and AB to process theleft-video plane and the right-video plane as follows. First, theparallax video generation unit 4710 etc. shifts the position of thepixel data in the left-video plane up by half the height HGT, i.e.HGT/2, and shifts the position of the pixel data in the right-videoplane down by HGT/2. The vertical center of the video image shown in thearea of the video planes other than the strips AT and AB thus matchesthe vertical center of the screen. In the left-video plane, half of thestrip AT is removed from the top, yielding an empty strip with a heightof HDT/2 at the bottom. In the right-video plane, half of the strip ABis removed from the bottom, yielding an empty strip with a height ofHDT/2 at the top. Next, the parallax video generation unit 4710 etcuniformly paints the strips a background color or black. In other words,the pixel data included in the strips is uniformly overwritten with datathat represents a background color or black.

FIG. 61C is a schematic diagram showing a left view LV represented bythe processed left-video plane and a right view RV represented by theprocessed right-video plane. As shown in FIG. 61C, the vertical centersof the left view LV and the right view RV match. Accordingly, thevertical position of the object OBJ shown in FIG. 61A is the same in theleft view LV and the right view RV. At the top of the left view LV, thestrip AT, which is included only in the vertical angle of view VAL ofthe left-video camera CML, is hidden by a black strip BT of heightHGT/2, and at the bottom of the right view RV, the strip AB, which isincluded only in the vertical angle of view VAR of the right-videocamera CMR, is hidden by a black strip BB of height HGT/2. Furthermore,a black strip BB of height HGT/2 is added to the bottom of the left viewLV, and a black strip BT of height HGT/2 is added to the top of theright view RV. As a result, both of the viewer's eyes see only the areashared by the left view LV and the right view RV, and the verticalpositions match between the object seen by each eye. This avoids therisk of causing the viewer to feel uncomfortable.

Alternatively, the parallax video generation unit 4710 etc. may processthe left-video plane and right-video plane as follows. First, viacropping similar to that shown in FIG. 49, the plane adder 4126 removesthe pixel data in the strips AT and AB from each of the video planes.Next, the parallax video generation unit 4710 etc. resizes each videoplane from the pixel data in the remaining area via scaling. The videoimage shown by the remaining area is thus expanded to fill the entireframe, and as a result, both of the viewer's eyes see only the areashared by the left view LV and the right view RV. Furthermore, thevertical positions match between the object seen by each eye. Thisavoids the risk of causing the viewer to feel uncomfortable.

(1-N-3) Misalignment of Graphics Images Between Left View and Right View

When a playback device in 1 plane+offset mode provides a large offset toa graphics plane to generate a pair of graphics planes, a region in theright or left edge of one graphics plane may not be included in theright or left edge of the other graphics plane.

FIG. 62A is a schematic diagram showing an example of graphics imagesrepresented by a graphics plane GPL. As shown in FIG. 62A, the graphicsplane GPL represents three types of graphic elements OB1, OB2, and OB3.In particular, the left edge of the left graphic element OB1 is locatedat a distance D1 from the left edge of the graphics plane GPL, and theright edge of the right graphic element OB3 is located at a distance D3from the right edge of the graphics plane GPL. FIGS. 62B and 62C areschematic diagrams respectively showing processes of providing a rightand left offset to the graphics plane GPL. As shown in FIG. 62B, a stripAR1 of width OFS equal to the offset value is removed from the rightedge of the graphics plane GPL, and a transparent strip AL1 of width OFSis added to the left edge, in a way similar to the cropping processshown in FIG. 49. The horizontal positions of the graphic elementsOB1-OB3 are thus shifted to the right from their original positions by adistance OFS equal to the offset value. On the other hand, as shown inFIG. 62B, a strip AL2 of width OFS equal to the offset value is removedfrom the left edge of the graphics plane GPL, and a transparent stripAR2 of width OFS is added to the right edge. The horizontal positions ofthe graphic elements OB1-OB3 are thus shifted to the left from theiroriginal positions by the distance OFS.

As shown in FIGS. 62B and 62C, the distance OFS, which is equal to theoffset value, is larger than the distance D1 between the left edge ofthe left graphic element OB1 and the left edge of the graphics planeGPL. The distance OFS is also larger than the distance D3 between theright edge of the right graphic element OB3 and the right edge of thegraphics plane GPL. Accordingly, a portion MP3 of the right edge of theright graphic element OB3 is missing in the graphics plane GP1 to whicha right offset has been provided. Also, a portion MP1 of the left edgeof the left graphic element OB1 is missing in the graphics plane GP2 towhich a left offset has been provided. However, the missing portion MP1of the left graphic element OB1 is included in the graphics plane GP1with the right offset, and the missing portion MP3 of the right graphicelement OB3 is included in the graphics plane GP2 with the left offset.As a result, these missing portions MP1 and MP3 are only seen by one ofthe viewer's eyes, which may make the viewer feel uncomfortable.

In the playback device 102, each of the cropping units 4731-4734 inplane adder 4226 refers to the offset information 4707 to perform offsetcontrol on the graphics plane GPL. At this point, each of the croppingunits 4731-4734 furthermore removes a strip that has a width equal tothe offset value and extends along the left or right edge of thegraphics plane GPL. In other words, the pixel data in the strip isoverwritten with data representing a transparent color. Alternatively,the output unit 5904 in the display device 103 may receive offsetinformation from the system target decoder 4225 or the signal processingunit 5903 in the display device 103 and refer to the offset informationto remove a strip from the left or right edge of the graphics plane GPLFIGS. 62B and 62C show the strips AS1 and AS2 to be removed. In thegraphics plane GP1 with the right offset, the strip AS1 to be removedincludes the missing portion MP1 of the left graphic element OB1. In thegraphics plane GP2 with the left offset, the strip AS2 to be removedincludes the missing portion MP3 of the right graphic element OB3.

FIGS. 62D and 62E are schematic diagrams showing graphics imagesrepresented by the graphics planes GP1 and GP2 with the right and leftoffsets, respectively. As shown in FIGS. 62D and 62E, in the graphicsplanes GP1 and GP2, the shapes of the three types of graphic elementsOB1-OB3 match. As a result, only the shared part of the graphics imagesare visible to each of the viewer's eyes. This avoids the risk ofcausing the viewer to feel uncomfortable.

Alternatively, the following condition may be prescribed regarding thearrangement of graphic elements for graphics planes played back from aPG stream or an IG stream on a BD-ROM disc and for a graphics planegenerated by a playback device 102. FIG. 63 is a schematic diagramshowing such a condition. As shown in FIG. 63, xy orthogonal coordinatesare established on the graphics plane GPL, with an origin (0, 0) at theupper-left corner. The x and y coordinates are respectively thehorizontal and vertical coordinates of the graphics plane GPL. Thecoordinates of the lower-right corner of the graphics plane GPL are setto (TWD, THG). Using these xy coordinates, the condition is set asfollows: in each frame, the graphic elements OB1, OB2, and OB3 must bepositioned within the rectangular area having four points (OFS, 0),(TWD-OFS, 0), (TWD-OFS, THG), and (OFS, THG) as vertices. In otherwords, graphic elements are prohibited from being placed within thestrips AL and AR of width OFS which respectively extend along the leftedge and right edge of the graphics plane GPL. As is clear from FIGS.62B and 62C, these strips AL and AR are removed by offset control.Accordingly, if graphic elements are prohibited from being placed withinthe strips AL and AR, the shapes of the graphic elements do not changeeven when an offset is provided to the graphics plane GPL. As a result,both of the viewer's eyes see the same graphics images, which avoids therisk of causing the viewer to feel uncomfortable.

(1-O) Letterbox Display

The screen size assumed at the time of authoring of a video contentdepends on the format of the content: Full-HD format adopted in digitalTV broadcasting; or Cinemascope™ format adopted in movies. The aspectratio of Full-HD is 16:9 (≈1.78:1), while the aspect ratio ofcinemascope is 2.35:1. Accordingly, in home movie contents recorded onBD-ROM discs, horizontal black strips are provided above and below eachvideo frame. The black strips are provided so that the whole aspectratio of the video frame and the black strips is adjusted to 16:9. Thisdisplay method is referred to as “letterbox display”.

FIGS. 64A1 and 64A2 are schematic diagrams showing the same screen inthe letterbox display. As shown in FIGS. 64A1 and 64A2, the resolutionof the whole screen is 1920×1080 pixels, and the aspect ratio is 16:9.On the other hand, the resolution of a display area DRG for displayingthe video images is 1920×818 pixels, and the aspect ratio is 2.35:9.Black strips BT and BB, each 131-pixel high, extend horizontally aboveand below the display area DRG.

When such a letterbox display is adopted in the display of 3D videoimages, it is preferable that the subtitle is displayed on either of theblack strips BT and BB, not on the display area DRG. This enables the 3Dvideo images and the subtitle to be separated from each other and bothto be presented to the viewer in a reliable manner. However, 131 pixelsof height of the black strips BT and BB is not necessarily sufficient todisplay the subtitle. In that case, the plane adders 4226 of theplayback device 102 provide the primary video plane with an offset inthe vertical direction. This offset control is referred to as “videoshift”. There are three types of video shifts: “Keep”, “Up”, and “Down”.In the Keep mode, the primary video plane is not provided with theoffset in the vertical direction. Thus as in the video frame shown inFIGS. 64A1 and 64A2, the height of each of black strips BT and BB iskept to be 131 pixels. In the Up mode, the primary video plane isprovided with an upward offset. FIG. 64B is a schematic diagram showingthe screen in which the primary video plane has been provided with anupward offset of 131 pixels. As shown in FIG. 64B, the black strip BThas been removed from the upper portion and the height of the blackstrip BB in the lower portion has been doubled. In the Down mode, theprimary video plane is provided with a downward offset. FIG. 64C is aschematic diagram showing the screen in which the primary video planehas been provided with a downward offset of 131 pixels. As shown in FIG.64C, the black strip BB has been removed from the lower portion and theheight of the black strip BT in the upper portion has been doubled. Inthis way, the plane adders 4226 increases either of the black strips BTand BB to a height sufficient to display the subtitle by executing thevideo shift in the Up or Down mode.

The size of the vertical offset may be other than 131 pixels. FIG. 64Dis a schematic diagram showing the screen in which the primary videoplane has been provided with an upward offset of 51 pixels. As shown inFIG. 64D, the height of the black strip BT in the upper portion has beendecreased to 131−51=80 pixels, and the height of the black strip BB inthe lower portion has been increased to 131+51=182 pixels. In thefollowing, it is assumed that the size of the offset is 131 pixels.

FIG. 65 is a functional block diagram showing the structure of theplayback device 102 required for the video shift. The structure shown inFIG. 65, in contrast to that shown in FIG. 47, includes a video shiftunit 6501, SPRM(32) 6502, and SPRM(33) 6503. The other components aresimilar. In FIG. 65, the components similar to ones shown in FIG. 47,are marked with the same reference numbers. Furthermore, details of thesimilar components can be found in the description on FIG. 47.

The values indicated by the SPRM(32) and SPRM(33) are set by the programexecution unit 4234 in accordance with an application program such as aBD-J object or a user's instruction via a GUI. The value indicated bythe SPRM(33) is further updated in accordance with the playlist file.

FIG. 66A is a table showing the data structures of the SPRM(32) andSPRM(33). As shown in FIG. 66A, the SPRM(32) stores a parameter thatindicates the video shift mode (video shift mode). The parameter cantake any of three values “0”, “1”, and “2” which correspond to threetypes of video shift modes. The SPRM(33) stores four pairs of a videoupward move shift value and a video downward move shift value. The shiftvalues are composed of: a pair for the PG plane (PG_shift_value_for_Up,PG_shift_value_for_Down); a pair for the IG plane(IG_shift_value_for_Up, IG_shift_value_for_Down); a pair for thesecondary video plane (SV_shift_value_for_Up, SV_shift_value_for_Down);and a pair for the image plane (IM_shift_value_for_Up,IM_shift_value_for_Down). Each of the video upward move shift value andthe video downward move shift value represents the size of an offset inthe vertical direction that is provided to the PG plane or the like whenan upward or downward offset is provided to the primary video plane.

FIG. 66B is a schematic diagram showing the STN table in the playlistfile for the video content of the letterbox display. As shown in FIG.66B, in the STN table 6600, an STN 6601 is associated with a streamentry 6602 of the PG stream 1 and stream attribute information 6603. Thestream attribute information 6603 includes a video upward move shiftvalue (PG_y_shift_value_for_Up) 6610 and a video downward move shiftvalue (PG_y_shift_value_for_Down) 6611. With this structure, these shiftvalues can be set for each PI. With regard to the other stream data suchas the IG stream, each shift value can be set. The playback control unit4235 reads a shift value from the STN table in each PI, and updates avalue indicated by the SPRM(33) with the read shift value.

The video shift unit 6501 receives the left-view plane data 4701 and theright-view plane data 4702 alternately from the switch 4720. Upon eachreception thereof, the video shift unit 6501 refers to the SPRM(32) inthe player variable storage unit 4236 and provides the primary videoplane with a vertical offset in a video shift mode indicated by thevalue in the SPRM(32). The video shift unit 6501 then transmits theprimary video plane to the second adder 4742.

FIGS. 67A-67C are schematic diagrams showing primary video planes VPA,VPB, and VPC processed by the video shift unit 6501 in the Up mode, Keepmode, and Down mode, respectively. When the SPRM(32) indicates the Keepmode, the video shift unit 6501 does not provide the primary video planewith a vertical offset. Thus as shown in FIG. 67B, the height of each ofblack strips BT and BB in the primary video plane VPB is kept to be 131pixels. When the SPRM(32) indicates the Up mode, the video shift unit6501, in a similar manner to the cropping process shown in FIG. 49,first cuts the black strip BT that is 131 pixels high out of the upperportion of the original primary video plane VPB. The video shift unit6501 then, as shown in FIG. 67A, adds a black strip AB that is 131pixels high to the lower portion of the primary video plane VPA. Thismoves the location of the pixel data other than the pixel data includedin the cut-out black strip BT upward by 131 pixels. On the other hand,the height of black strips BB+AB is increased to 131×2=262 pixels. Whenthe SPRM(32) indicates the Down mode, the video shift unit 6501 cuts theblack strip BB that is 131 pixels high out of the lower portion of theoriginal primary video plane VPB, and as shown in FIG. 67C, adds a blackstrip AT that is 131 pixels high to the upper portion of the primaryvideo plane VPC. This moves the location of the pixel data other thanthe pixel data included in the cut-out black strip downward by 131pixels. On the other hand, the height of black strips BT+AT is increasedto 131×2=262 pixels.

Referring again to FIG. 65, each time it receives the PG plane data 4704from the system target decoder 4225, the second cropping unit 4732refers to the SPRM(32) 6502 and the SPRM(33) 6503 and provides the PGplane 4704 with a vertical offset in accordance with values stored inthe SPRMs. Furthermore, in the 1 plane+offset mode, the second croppingunit 4732 provides the PG plane 4704 with a horizontal offset. Thesecond cropping unit 4732 then transmits the PG plane 4704 to the secondadder 4742.

FIGS. 67D-67F are schematic diagrams showing PG planes PGD, PGE, and PGFprocessed by the second cropping unit 4732 in the Up mode, Keep mode,and Down mode, respectively. When the SPRM(32) indicates the Keep mode,the second cropping unit 4732 does not provide the PG plane PGE with avertical offset. Thus as shown in FIG. 67E, the subtitle SUB in the PGplane PGE is kept to be at the original position. When the SPRM(32)indicates the Up mode, the second cropping unit 4732 first reads a videoupward move shift value (PG_shift_value_for_Up) “a” for the PG planefrom the SPRM(33) 6503. The second cropping unit 4732 then, in a similarmanner to the cropping process shown in FIG. 49, provides the PG planePGE with a downward offset which has the same size as the video upwardmove shift value “a”. More specifically, the second cropping unit 4732first cuts the strip SBE that is “a” pixels high out of the lowerportion of the original PG plane PGE. The second cropping unit 4732then, as shown in FIG. 67D, adds a strip STD that is “a” pixels high tothe upper portion of the PG plane PGD. This moves the location of thesubtitle SUB downward by “a” pixels. When the SPRM(32) indicates theDown mode, the second cropping unit 4732 first reads a video downwardmove shift value (PG_shift_value_for_Down) “b” for the PG plane from theSPRM(33) 6503. The second cropping unit 4732 then, in a similar mannerto the cropping process shown in FIG. 49, provides the PG plane PGE withan upward offset which has the same size as the video downward moveshift value “b”. More specifically, the second cropping unit 4732 firstcuts the strip STE that is “b” pixels high out of the upper portion ofthe original PG plane PGE. The second cropping unit 4732 then, as shownin FIG. 67F, adds a strip SBF that is “b” pixels high to the lowerportion of the PG plane PGF. This moves the location of the subtitle SUBupward by “b” pixels.

The second adder 4742 receives the PG plane data from the secondcropping unit 4732, superimposes the PG plane data on the primary videoplane data from the video shift unit 6501 and transmits the result tothe third adder 4743. FIGS. 67G-67I are schematic diagrams showing planedata PLG, PLH, and PLI combined by the second adder 4742 in the Up mode,Keep mode, and Down mode, respectively. In the Keep mode, as shown inFIG. 67H, the subtitle SUB is displayed on top of the primary videoimages MVW. In the Up mode, as shown in FIG. 67G, the subtitle SUB isdisplayed in the black strip BBG that is located below the primary videoimages MVW. This can be realized by adjusting the video upward moveshift value “a”. In the Down mode, as shown in FIG. 671, the subtitleSUB is displayed in the black strip BTI that is located above theprimary video images MVW. This can be realized by adjusting the videodownward move shift value “b”.

In the letterbox display, a dialog screen represented by the IG plane,video images represented by the secondary video plane, or a pop-up menurepresented by the image plane may be displayed in the black strip, aswell as the subtitle represented by the PG plane. In those cases, theheight of the black strip can be adjusted appropriately by a methodsimilar to the above-described method.

(1-O-1) In the structure shown in FIG. 65, the second cropping unit 4732reads the video upward/downward move shift value from the SPRM(33) 6503.Alternatively, the second cropping unit 4732 may read the videoupward/downward move shift value directly from the playlist file.

(1-O-2) The height of the black strips BT and BB may be other than 131pixels, and may further be variable. The value thereof may be set in anSPRM in the player variable storage unit 4236 in accordance with anapplication program or the user.

(1-O-3) In FIGS. 67D and 67F, the second cropping unit 4732 moves thelocation of almost all pixel data included in the PG plane PGE upwardand downward. Alternatively, the PG decoder may change the objectdisplay position indicated by the PCS by referring to the SPRM(33) 6503.For example, when the PCS indicates the object display position=(x, y)and the SPRM(33) 6503 indicates the video upward move shift value “a”,the PG decoder changes the object display position to coordinates (x,y+a). With this operation, like the subtitle SUB shown in FIG. 67D, thegraphics object represented by the PG stream is displayed below theobject display position indicated by the PCS. This also applies to thecase where the display position of the graphics object is moved upward.Note that the PCS may store the video upward/downward move shift value.

(1-O-4) In the Up mode and Down mode, as shown in FIGS. 67D and 67F, theupper and lower portions of the PG plane are cut out. At this point, inorder to prevent the upper and lower portions of the graphics objectfrom being cut out, the area in which the graphics object can bearranged may be limited to a predetermined range. As a specific example,it is assumed that the height×width of the PG plane is HGT×WDH, thevideo upward move shift value is “a”, and the video downward move shiftvalue is “b”. In that case, as shown in FIG. 67E, the arrangement of thegraphics object is limited to within the following horizontal strip: thex-y coordinates of the upper-left corner PUL=(0, b); and the x-ycoordinates of the lower-left corner PDR=(WDH, HGT−a). More accurately,the PG stream satisfies the following conditions: (A) the object displayposition indicated by the PCS is within the above-described strip; (B)even if the graphics object is displayed at the object display position,the display area does not exceed the range of the above-described strip;(C) the window position indicated by the WDS is within theabove-described strip; and (D) even if the window is set at the windowposition, the range thereof does not exceed the range of theabove-described strip. In this way, it is possible to prevent the upperand lower portions of the graphics object from being cut out.

(1-O-5) FIG. 68A is a schematic diagram showing another example of theSTN table in the playlist file for the video content of the letterboxdisplay. As shown in FIG. 68A, in the STN table 6800, an STN 6801 isassociated with a stream entry 6802 of the PG stream 1 and streamattribute information 6803. The stream attribute information 6803includes a video shift mode 6812 as well as a video upward/downward moveshift value 6810, 6811. In that case, the playback device 102 may usethe following structure for the video shift.

FIG. 69 is a functional block diagram showing another example of thestructure of the playback device 102 required for the video shift. Thestructure shown in FIG. 69 differs from the structure shown in FIG. 65in the video shift unit 6901 and SPRM(34) 6904. The other components aresimilar. In FIG. 69, the components similar to ones shown in FIG. 65 aremarked with the same reference numbers. Furthermore, details of thesimilar components can be found in the description on FIG. 65.

As shown in FIG. 66A, the SPRM(32) represents the video shift mode, andthe SPRM(33) represents the video upward move shift value and the videodownward move shift value. The parameters representing those are updatedin accordance with the STN table in the playlist file as shown in FIG.68A. In the player variable storage unit 4236, the SPRM(34) furtherstores a flag whose ON/OFF indicates whether the video shift is to beperformed. The value of the flag is set by the program execution unit4234 in accordance with an application program or the user. Each time itreceives either the left-video plane data 4701 or the right-video planedata 4702 from the switch 4720, the video shift unit 6901 first refersto the flag in the SPRM(34) to determine whether to perform the videoshift. For example, when the value of the flag is “1”, the video shiftunit 6901 refers to the SPRM(32) and provides the primary video planewith a vertical offset in a video shift mode indicated by the value inthe SPRM(32). On the other hand, when the value of the flag is “0”, thevideo shift unit 6901 transmits the primary video plane to the secondadder 4742 without performing the video shift. Similarly, each time itreceives the PG plane data 4704 from the system target decoder 4225, thesecond cropping unit 4732 first refers to the SPRM(34) to determinewhether to provide the PG plane 4704 with a vertical offset. Forexample, when the value of the flag is “1”, the second cropping unit4732 refers to the SPRM(32) and SPRM(33) and provides the PG plane 4704with a vertical offset in accordance with the values therein. On theother hand, when the value of the flag is “0”, the second cropping unit4732 does not provide the PG plane 4704 with a vertical offset.

(1-O-6) When a plurality of pieces of stream attribute information 6803,each including the video shift mode 6812 shown in FIG. 68A, areregistered in the STN table, the order of registration is set so thatpieces of stream attribute information having the same video shift modebecome continuous. FIG. 68B is a schematic diagram showing the order ofregistration. As shown in FIG. 68B, PIDs of nine PG streams 1-9 areregistered in the STN table, in correspondence with stream numbers(STNs) 5-13. The video shift mode of PG streams 1-3 is set to the Keepmode, the video shift mode of PG streams 4 and 5 is set to the Up mode,and the video shift mode of PG streams 6-9 is set to the Down mode. Inthat case, continuous three STNs=1-3 are assigned to the PG streams 1-3,continuous two STNs=4, 5 are assigned to the PG streams 4, 5, andcontinuous four STNs=6-9 are assigned to the PG streams 6-9. Each timeit receives notification of depression of a subtitle switch button fromthe remote control 105, the playback device 102 selects a PG stream tobe used in the display of subtitle from among PG stream 1-9 inaccordance with the registration order indicated in FIG. 68B. Here,since the screen display of both the video images and the subtitle iscontinued during the selection operation, generally the displaypositions of the video images and the subtitle change when the subtitleswitch button is pressed. However, as shown in FIG. 68B, PG streams ofthe same video shift mode are continuously registered in the STN table.Thus generally the display positions of the video images and thesubtitle change only after the subtitle switch button is pressed aplurality of times. In this way, the frequency of change is suppressed,thereby preventing the selection operation of the PG stream fromdisrupting the display of the video images and the subtitle.

(1-O-7) When switching between the video shift modes, the playbackdevice 102 may change the display position of the video images andsubtitle smoothly by using the visual effects such as the fade-in/out.More preferably, the change of display position of the subtitle isdelayed than the change of display position of the video images. Thisprevents the risk that the change of display position of the videoimages and subtitle due to a switch between video shift modes may causeviewers to feel uncomfortable.

(1-O-8) In the PDS in the PG stream, “colorless transparent” is assignedto color ID=255, and in the WDS, color ID=255 is assigned to thebackground color in the window. Accordingly, when the PG streamrepresents the subtitle, the background color of the subtitle is set tothe colorless transparent. FIG. 70B is a schematic diagram showing thevideo image IMG and subtitle SUB displayed on the screen SCR in thatcase. As shown in FIG. 70B, in the window WIN1 indicating the displayrange of the subtitle SUB, the background color is colorlesstransparent. Accordingly, in the window WIN1, the video image IMG andsubtitle SUB are displayed on top of each other.

On the other hand, the playback device 102 may assign an opaque colorsuch as black to color ID=255. FIG. 70A is a schematic diagram showingthe data structure of the SPRM(37) in the player variable storage unit4236. As shown in FIG. 70A, a color coordinate value of the backgroundcolor of the subtitle is stored in the SPRM(37). The value is preset bythe program execution unit 4234 in accordance with an applicationprogram or the user. When a color coordinate value is set in theSPRM(37), the PG decoder in the system target decoder 4225 assigns thecolor coordinate value to color ID=255 irrespectively of the settingindicated by the PDS. FIG. 70C is a schematic diagram showing the videoimage IMG and subtitle SUB displayed on the screen SCR in that case.Here, the color coordinate value indicated by the SPRM(37) is an opaquecolor such as black. As shown in FIG. 70C, in the window WIN2 indicatingthe display range of the subtitle SUB, the background color is theopaque color. Accordingly, in the window WIN2, the video image IMG ishidden by the background color, and only the subtitle SUB is displayed.In this way, it is possible to present the video images and the subtitleto the viewer in a reliable manner.

(1-O-9) With a switch between video shift modes, not only the displayposition of the graphics image represented by the PG stream but the PGstream itself may be changed. FIG. 71A is a schematic diagram showing afurther another example of the STN table in the playlist file for thevideo content of the letterbox display. As shown in FIG. 71A, in the STNtable 7100, an STN 7101 is associated with a stream entry 7102 of the PGstream 1 and stream attribute information 7103. The stream attributeinformation 7103 includes a video upward/downward move shift value 7110,7111. The video upward move shift value 7110 and the video downward moveshift value 7111 indicates PIDs of the PG streams that are to beselected when the Up mode and the Down mode are selected as the videoshift mode, respectively. In the PG streams indicated by the videoupward move shift value 7110 and the video downward move shift value7111, initially the display position of the subtitle is set in eachblack strip of the lower and upper portions of the primary video plane.In that case, the playback device 102 may use the following structurefor the video shift.

FIG. 71B is a functional block diagram showing a further another exampleof the structure of the playback device 102 required for the videoshift. The structure shown in FIG. 71B differs from the structure shownin FIG. 65 in the following points: (A) each time the video shift modeis switched, the playback control unit 7135 specifies the PID of the PGstream to be selected newly to the PG decoder 4072; (B) the secondcropping unit 4732 does not provide the PG plane 4704 with a verticaloffset; (C) the SPRM(33) and SPRM(34) may not be set in the playervariable storage unit 4236. The other components are the same. Thus inFIG. 71B, the components that are also shown in FIG. 65 are assigned thesame reference numbers. Furthermore, details on these components can befound in the description of FIG. 65.

The video shift mode indicated by the SPRM(32) 6502 is changed by theprogram execution unit 4234 in accordance with an application program orthe user. Each time it detects a change in the value stored in theSPRM(32) 6502, the playback control unit 7135 refers to the STN tableshown in FIG. 71A. By doing so, the playback control unit 7135 retrievesthe PID of the PG stream corresponding to the video shift mode after thechange, and passes the PID to the PG decoder 4072. More specifically,when the SPRM(32) 6502 indicates the Up mode, the playback control unit7135 retrieves the PID indicated by the video upward move subtitle 7110;when the SPRM(32) 6502 indicates the Down mode, the playback controlunit 7135 retrieves the PID indicated by the video downward movesubtitle 7111; and when the SPRM(32) 6502 indicates the Keep mode, theplayback control unit 7135 retrieves the PID indicated by the streamentry 7102. As a result, the PG plane 4704 decoded by the PG decoder4072 represents a subtitle that corresponds to the video shift mode.

FIG. 72A is a schematic diagram showing the subtitles SB1 and SB2 thatcorrespond to the Keep mode. FIG. 72B is a schematic diagram showing thesubtitles SB1 and SB2 that correspond to the Down mode. As shown in FIG.72A, in the Keep mode, a horizontal subtitle SB1 is displayed on top ofthe lower portion of the video image display area VP1, and a verticalsubtitle SB2 is displayed on top of the right-end portion of the videoimage display area VP1. Furthermore, the black strips BT and BB that are131 pixels high are displayed in the portions above and below the videoimage display area VP1, respectively. As shown in FIG. 72B, in the Downmode, the black strip BT2 that is 262 pixels high is displayed in theportion above the video image display area VP2. If the display positionof the horizontal subtitle SB1 were moved into the black strip BT2 ofthe upper portion by providing the PG plane with a vertical offset, thedisplay position of the vertical subtitle SB2 would be moved upwardoutside the screen to the display position SB20. On the other hand, thePG stream indicated by the video downward move subtitle 7111 representsthe horizontal subtitle SBD, and the display position thereof has beenset in the black strip BT2 in upper portion in advance. Accordingly, theplayback control unit 7135 changes the PG stream that represents thehorizontal subtitle SB1 in the Keep mode to the PG stream indicated bythe video downward move subtitle 7111 in the Down mode. On the otherhand, the playback control unit 7135 uses the PG stream, whichrepresents the vertical subtitle SB2 in the Keep mode, as it is in theDown mode. With this structure, in the Down mode, as shown in FIG. 72B,the horizontal subtitle SBD is displayed in the black strip BT2 in theupper portion, and the vertical subtitle SB2 is displayed, as in theKeep mode, on top of the right-end portion of the video image displayarea VP1. This also applies to the Up mode.

When the video upward move subtitle 7110, the video downward movesubtitle 7111, or the stream entry 7102 is not registered in the STNtable, no new PID is specified by the playback control unit 7135, thusthe PG decoder 4072 maintains the PID held at this point as it is. Inthat case, the second cropping unit 4732 may provide the PG plane 4704with a vertical offset. This offset is the same as the offset providedby the video shift unit 6501 to the primary video planes 4701, 4702.FIG. 72C is a schematic diagram showing the subtitle SB1 displayed inthe Keep mode. FIG. 72D is a schematic diagram showing the subtitle SB3displayed in the Up mode when the video upward subtitle 7110 is notregistered in the STN table. As shown in FIG. 72C, in the Keep mode, thesubtitle SB1 is displayed on top of the lower portion of the video imagedisplay area VP1. Furthermore, the black strips BT and BB that are 131pixels high are displayed in the portions above and below the videoimage display area VP1, respectively. As shown in FIG. 72D, in the Upmode, the black strip BB2 that is 262 pixels high is displayed in theportion below the video image display area VP2. If the display positionof the horizontal subtitle SB1 were maintained to the position in theKeep mode, the lower portion of the subtitle SB1 would be displayed ontop of the black strip BB2. In contrast, when the second cropping unit4732 provides the PG plane 4704 with a vertical offset, as shown in FIG.72D, in the Up mode, the subtitle SB3 is displayed in the lower portionof the video image display area VP2 at a position separate from theblack strip BB2 in a similar manner to the subtitle SB1 in the Keepmode.

Embodiment 2

The BD-ROM disc according to Embodiment 2 of the present invention alsoincludes a pair of a base view and a dependent view for the PG streamand the IG stream. On the other hand, the playback device according toEmbodiment 2 of the present invention is provided with 2 plane mode. “2plane mode” is one of the display modes for the graphics plane. When asub-TS includes both a base-view and dependent-view graphics stream, theplayback device in 2 plane mode decodes and alternately outputsleft-view and right-view graphics plane data from the graphics streams.3D graphics images can thus be played back from the graphics streams.Apart from these points, the BD-ROM disc and playback device accordingto Embodiment 2 have the same structure and functions as according toEmbodiment 1. Accordingly, the following is a description of the BD-ROMdisc and playback device according to Embodiment 2 insofar as these havebeen changed or expanded as compared to Embodiment 1. Details on theparts of the BD-ROM disc and playback device that are the same asaccording to Embodiment 1 can be found in the description of Embodiment1.

<Data Structure of Sub-TS>

FIG. 73A is a list of elementary streams multiplexed in a first sub-TSon a BD-ROM disc 101. The first sub-TS is multiplexed stream data inMPEG-2 TS format and is included in a file DEP for the L/R mode. Asshown in FIG. 73A, the first sub-TS includes a primary video stream7311, left-view PG streams 7312A and 7312B, right-view PG streams 7313Aand 7313B, left-view IG stream 7314, right-view IG stream 7315, andsecondary video stream 7316. When the primary video stream 301 in themain TS shown in FIG. 3A represents the left view of 3D video images,the primary video stream 7311, which is a right-view video stream,represents the right view of the 3D video images. The pairs of left-viewand right-view PG streams 7312A+7313A and 7312B+7313B represent the leftview and right view of graphics images, such as subtitles, when thesegraphics images are displayed as 3D video images. The pair of left-viewand right-view IG streams 7314 and 7315 represent the left view andright view of graphics images for an interactive screen when thesegraphics images are displayed as 3D video images. When the secondaryvideo stream 306 in the main TS represents the left view of 3D videoimages, the secondary video stream 7316, which is a right-view videostream, represents the right view of the 3D video images.

PIDs are assigned to the elementary streams 7311-7316 as follows, forexample. A PID of 0x1012 is assigned to the primary video stream 7311.When up to 32 other elementary streams can be multiplexed by type in onesub-TS, the left-view PG streams 7312A and 7312B are assigned any valuefrom 0x1220 to 0x123F, and the right-view PG streams 7313A and 7313B areassigned any value from 0x1240 to 0x125F. The left-view IG stream 7314is assigned any value from 0x1420 to 0x143F, and the right-view IGstream 7315 is assigned any value from 0x1440 to 0x145F. The secondaryvideo stream 7316 is assigned any value from 0x1B20 to 0x1B3F.

FIG. 73B is a list of elementary streams multiplexed in a second sub-TSon a BD-ROM disc 101. The second sub-TS is multiplexed stream data inMPEG-2 TS format and is included in a file DEP for the depth mode.Alternatively, the second sub-TS may be multiplexed in the same file DEPas the first sub-TS. As shown in FIG. 73B, the second sub-TS includes aprimary video stream 7321, depth map PG streams 7323A and 7323B, depthmap IG stream 7324, and secondary video stream 7326. The primary videostream 7321 is a depth map stream and represents 3D video images incombination with the primary video stream 301 in the main TS. When the2D video images represented by the PG streams 323A and 323B in the mainTS are used to project 3D video images on a virtual 2D screen, the depthmap PG streams 7323A and 7323B are used as the PG streams representing adepth map for the 3D video images. When the 2D video images representedby the IG stream 304 in the main TS are used to project 3D video imageson a virtual 2D screen, the depth map IG stream 7324 is used as the IGstream representing a depth map for the 3D video images. The secondaryvideo stream 7326 is a depth map stream and represents 3D video imagesin combination with the secondary video stream 306 in the main TS.

PIDs are assigned to the elementary streams 7321-7326 as follows, forexample. A PID of 0x1013 is assigned to the primary video stream 7321.When up to 32 other elementary streams can be multiplexed by type in onesub-TS, the depth map PG streams 7323A and 7323B are assigned any valuefrom 0x1260 to 0x127F. The depth map IG stream 7324 is assigned anyvalue from 0x1460 to 0x147F. The secondary video stream 7326 is assignedany value from 0x1B40 to 0x1B5F.

<Data Structure of STN Table SS>

FIG. 74 is a schematic diagram showing a data structure of the STN tableSS 3130 according to Embodiment 2 of the present invention. As shown inFIG. 74, the stream registration information sequences 3301, 3302, 3303,. . . in the STN table SS 3130 include a stream registration informationsequence 7413 of a PG stream and a stream registration informationsequence 7414 of an IG stream in addition to an offset during pop-up3311 and a stream registration information sequence 3312 of adependent-view video stream that are shown in FIG. 33.

The stream registration information sequence 7413 of a PG streamincludes stream registration information indicating the PG streams thatcan be selected for playback from the sub-TS. The stream registrationinformation sequence 7414 of an IG stream includes stream registrationinformation indicating the IG streams that can be selected for playbackfrom the sub-TS. These stream registration information sequences 7413and 7414 are used in combination with the stream registrationinformation sequences, included in the STN table of the correspondingPI, that indicate PG streams and IG streams. When reading a piece ofstream registration information from an STN table, the playback device102 in 3D playback mode automatically also reads the stream registrationinformation sequence, located in the STN table SS, that has beencombined with the piece of stream registration information. When simplyswitching from 2D playback mode to 3D playback mode, the playback device102 can thus maintain already recognized STNs and stream attributes suchas language.

As further shown in FIG. 74, the stream registration informationsequence 7413 of the PG stream generally includes a plurality of piecesof stream registration information 7431. These are the same in number asthe pieces of stream registration information in the corresponding PIthat indicates the PG streams. The stream registration informationsequence 7414 of the IG stream includes the same sort of pieces ofstream registration information. These are the same in number as thepieces of stream registration information in the corresponding PI thatindicates the IG streams.

Each piece of stream registration information 7431 includes an STN 7441,stereoscopic flag (is_SS_PG) 7442, base-view stream entry(stream_entry_for_base_view) 7443, dependent-view stream entry(stream_entry_for_dependent_view) 7444, and stream attribute information7445. The STN 7441 is a serial number assigned individually to pieces ofstream registration information 7431 and is the same as the STN assignedto the piece of stream registration information, located in thecorresponding PI, with which the piece of stream registrationinformation 7431 is combined. The stereoscopic flag 7442 indicateswhether a BD-ROM disc 101 includes both base-view and dependent-view PGstreams. If the stereoscopic flag 7442 is on, the sub-TS includes bothPG streams. Accordingly, the playback device reads all of the fields inthe base-view stream entry 7443, the dependent-view stream entry 7444,and the stream attribute information 7445. If the stereoscopic flag 7442is off, the playback device ignores all of these fields 7443-7445. Boththe base-view stream entry 7443 and the dependent-view stream entry 7444include sub-path ID reference information 7421, stream file referenceinformation 7422, and PIDs 7423. The sub-path ID reference information7421 indicates a sub-path ID of a sub-path that specifies the playbackpaths of the base-view and dependent-view PG streams. The stream filereference information 7422 is information to identify the file DEPstoring the PG streams. The PIDs 7423 are the PIDs for the PG streams.The stream attribute information 7445 includes attributes for the PGstreams, such as language type.

Note that the stream registration information 7431 of the PG stream maybe stored in the STN table instead of the STN table SS. In that case,the stream registration information 7431 is stored in PG streams in themain TS, in particular in the stream attribute information therein.

<System Target Decoder>

FIG. 75 is a functional block diagram of a system target decoder 7525according to Embodiment 2 of the present invention. As shown in FIG. 75,the PG decoder 7501 supports 2 plane mode, unlike the PG decoder shownin FIG. 45. Specifically, the PG decoder 7501 includes a base-view PGdecoder 7511 and a dependent-view PG decoder 7512. In addition todecoding the PG streams 303A and 303B in the main TS shown in FIG. 3A,the base-view PG decoder 7511 decodes the left-view PG streams 7312A and7312B in the first sub-TS shown in FIG. 73A into plane data. Thedependent-view PG decoder 7512 decodes the right-view PG streams 7313Aand 7313B in the first sub-TS shown in FIG. 73A and the depth map PGstreams 7323A and 7323B in the second sub-TS shown in FIG. 73B intoplane data. The secondary video decoder and the IG decoder both includea similar pair of decoders. The system target decoder 7525 furtherincludes a pair of PG plane memories 7521 and 7522. The base-view PGdecoder 7511 writes the plane data into the left PG plane memory 7521,and the dependent-view PG decoder 7512 writes the plane data into theright PG plane memory 7522. The IG plane memory and the image planememory both have similar structures. The system target decoder 7525further associates the output of plane data from the graphics planememory with 2 plane mode, 1 plane+offset mode, and 1 plane+zero offsetmode. In particular, when the playback control unit 4235 indicates 2plane mode, the system target decoder 7525 alternately outputs planedata from a pair of PG plane memories 7521 and 7522 to the plane adder7526.

<Plane Adders>

FIG. 76 is a partial functional block diagram of the plane adder 7526 in2 plane mode. As shown in FIG. 76, the plane adder 6226 includes aparallax video generation unit 4710, switch 4720, first adder 4741, andsecond adder 4742, like the plane adder 4226 shown in FIG. 47. The planeadder 7526 further includes a second parallax video generation unit 7610and a second switch 7620 as units for input of PG plane data 7604 and7605. A similar structure is included in the units for input ofsecondary video plane data, IG plane data, and image plane data.

The second parallax video generation unit 7610 receives left PG planedata 7604 and right PG plane data 7605 from the system target decoder7525. In the playback device 102 in L/R mode, the left PG plane data7604 represents the left-view PG plane, and the right PG plane data 7605represents the right-view PG plane. At this point, the second parallaxvideo generation unit 7610 transmits the pieces of plane data 7604 and7605 as they are to the switch 7620. On the other hand, in the playbackdevice 102 in depth mode, the left PG plane data 7604 represents the PGplane of 2D graphics images, and the right PG plane data 7605 representsa depth map corresponding to the 2D graphics images. In this case, thesecond parallax video generation unit 7610 first calculates thebinocular parallax for each element in the 2D graphics images using thedepth map. Next, the second parallax video generation unit 7610processes the left PG plane data 7604 to shift the presentation positionof each element in the 2D graphics image in the PG plane to the left orright in accordance with the calculated binocular parallax. Thisgenerates a pair of PG planes representing a left view and right view.Furthermore, the second parallax video generation unit 7610 outputs thispair of PG planes to the second switch 7620.

The second switch 7620 outputs the left PG plane data 7604 and the rightPG plane data 7605, which have the same PTS, to the second adder 4742 inthis order. The second cropping unit 4732 in the 2 plane mode outputseach of the PG plane data 7604 and 7605 to the second adder 4742 as itis. The second adder 4742 superimposes each of the PG plane data 7604and 7605 on the plane data from the first adder 4741 and transmits theresult to the third adder 4743. As a result, the left-view PG plane issuperimposed on the left-video plane data 7601, and the right-view PGplane is superimposed on the right-video plane data 7602.

The second cropping unit 4732 in the 2 plane mode, in a similar mannerto the second cropping unit 4732 in the 1 plane+offset mode shown inFIG. 47, may provide each of PG plane data 7604 and 7605 with ahorizontal offset by using the offset adjustment value. This enables thedepth of the 3D graphics video images to be adjusted in conjunction withthe screen size of the display device 103. Alternatively, the secondcropping unit 4732, in a similar manner to the second cropping unit 4732in the 1 plane+offset mode shown in FIGS. 65 and 69, may provide each ofPG plane data 7604 and 7605 with a vertical offset by using the videoupward/downward move shift value. When a 3D video image is presented ina letterbox display, this enables a 3D graphics image (e.g. subtitle) tobe presented in a black strip which is located above or below the 3Dvideo image.

<Use of Offset Information in 2 Plane Mode>

The second cropping unit 4732 in the 2 plane mode may perform an offsetcontrol onto the left-view or right-view graphics plane by using theoffset information 4704. The offset control provides the followingadvantages.

In the L/R mode, instead of the left-view PG stream in the first sub-TSshown in FIG. 73A, the PG stream for 2D image in the main TS(hereinafter, the PG stream for 2D image is abbreviated as “2D PGstream”) may be used as the left-view PG plane data. That is to say, inthe base-view stream entry 7443 shown in FIG. 74, the sub-path IDreference information 7421 indicates the main path, the stream filereference information 7422 indicates the file 2D which stores the 2D PGstream, and the PID 7423 indicates the PID of the 2D PG stream. In thatcase, it is possible to reduce the data amount of the 3D video contentbecause the first sub-TS does not need to include the left-view PGstream. On the other hand, the following errors may occur to the 3Dgraphics image.

FIGS. 77A, 77B, and 77C are schematic diagrams showing a left-viewgraphics image GOB0 represented by the 2D PG stream and a right-viewgraphics images GOB1-GOB3 represented by the right-view PG stream. InFIGS. 77A, 77B, and 77C, the solid line in the screen SCR indicates theleft-view graphics image GOB0 and the dashed line indicates theright-view graphics images GOB1-GOB3. As the distances Δ1, Δ2, and Δ3between the graphics images shown in FIGS. 77A, 77B, and 77C change fromsmall to large in this order (Δ1<Δ2<Δ3), the difference in depth betweenthe 3D graphics image and the screen SCR changes accordingly. Thus whenthe pair of 3D graphics images is displayed as shown in FIG. 77A to FIG.77C in this order, the 3D graphics image appears to jump from the screenSCR toward the viewer. When the left-view graphics image GOB0 representsa subtitle, the image GOB0 is used as a 2D image as well, thus thedisplay position is constant through FIGS. 77A, 77B, and 77C. On theother hand, the display positions of the right-view graphics imagesGOB1-GOB3 move leftward in the order of FIGS. 77A, 77B, and 77C.Accordingly, the center positions C1, C2, and C3 between the graphicsimages move leftward in the order of FIGS. 77A, 77B, and 77C. That is tosay, the 3D graphics image of the subtitle appears to move leftward.Such a move of the subtitle may make the viewer feel uncomfortable.

The second cropping unit 4732 in the 2 plane mode prevents the 3Dgraphics image from moving horizontally by using the offset controlaccording to the offset information as follows. FIGS. 77D, 77E, and 77Fare schematic diagrams showing the offset control performed onto theleft-view graphics image shown in FIGS. 77A, 77B, and 77C. In FIGS. 77D,77E, and 77F, the solid line in the screen SCR indicates the left-viewgraphics images GOB4-GOB6 after the offset control, the thin dashed lineindicates the left-view graphics image GOB0 before the offset control,and the thick dashed line indicates the right-view graphics imagesGOB1-GOB3. The second cropping unit 4732 provides the left-view PG planewith offsets OFS1, OFS2, and OFS3 indicated by arrows in FIGS. 77D, 77E,and 77F in this order. With this provision, the left-view graphicsimages GOB4-GOB6 after the offset control move rightward compared to theleft-view graphics image GOB0 before the offset control. As a result,since the center position C0 between the graphics images is keptconstant through FIGS. 77D-77F, the 3D graphics image appears not tomove horizontally. In this way, by using the 2D PG stream as theleft-view PG stream, it is possible to prevent the viewer from feelinguncomfortable.

Embodiment 3

The following describes, as Embodiment 3 of the present invention, adevice and method for recording data on the recording media ofEmbodiments 1 and 2 of the present invention. The recording devicedescribed here is called an authoring device. The authoring device isgenerally located at a creation studio and used by authoring staff tocreate movie content to be distributed. First, in response to operationsby the authoring staff, the recording device converts movie content intoAV stream files using a predetermined compression encoding method. Next,the recording device generates a scenario. A “scenario” is informationdefining how each title included in the movie content is to be playedback. Specifically, a scenario includes dynamic scenario information andstatic scenario information. Then, the recording device generates avolume image for a BD-ROM disc from the AV stream files and scenario.Lastly, the recording device records the volume image on the recordingmedium.

FIG. 78 is a functional block diagram of a recording device 7800. Asshown in FIG. 78, the recording device 7800 includes a database unit7801, video encoder 7802, material creation unit 7803, scenariogeneration unit 7804, BD program creation unit 7805, multiplexprocessing unit 7806, and format processing unit 7807.

The database unit 7801 is a nonvolatile storage device embedded in therecording device and is in particular a hard disk drive (HDD).Alternatively, the database unit 7801 may be an external HDD connectedto the recording device, or a nonvolatile semiconductor memory deviceinternal or external to the recording device.

The video encoder 7802 receives video data, such as uncompressed bit mapdata, from the authoring staff and compresses the received video data inaccordance with a compression encoding method such as MPEG-4 AVC orMPEG-2. This process converts primary video data into a primary videostream and secondary video data into a secondary video stream. Inparticular, 3D video image data is converted into a pair of a base-viewvideo stream and a dependent-view video stream, as shown in FIG. 7,using a multiview coding method such as MVC. In other words, the videoframe sequence representing the left view is converted into a base-viewvideo stream via inter-picture predictive encoding on the pictures inthese video frames. On the other hand, the video frame sequencerepresenting the right view is converted into a dependent-view videostream via predictive encoding on not only the pictures in these videoframes, but also the base-view pictures. Note that the video framesrepresenting the right view may be converted into a base-view videostream, and the video frames representing the left view may be convertedinto a dependent-view video stream. The converted video streams 7812 arestored in the database unit 7801.

During the process of inter-picture predictive encoding, the videoencoder 7802 detects motion vectors between individual images in theleft view and right view and calculates depth information of each 3Dvideo image based on the detected motion vectors. FIGS. 79A and 79B areschematic diagrams respectively showing a picture in a left view and aright view used to display one scene of 3D video images, and FIG. 79C isa schematic diagram showing depth information calculated from thesepictures by the video encoder 7802.

The video encoder 7802 compresses left-view and right-view picturesusing the redundancy between the pictures. In other words, the videoencoder 7802 compares both uncompressed pictures on a per-macroblockbasis, i.e. per matrices of 8×8 or 16×16 pixels, so as to detect amotion vector for each image in the two pictures. Specifically, as shownin FIGS. 79A and 79B, a left-view picture 7901 and a right-view picture7902 are first each divided into macroblocks 7903. Next, the areasoccupied by the image data in picture 7901 and picture 7902 are comparedfor each macroblock 7903, and a motion vector for each image is detectedbased on the result of the comparison. For example, the area occupied byimage 7904 showing a “house” in picture 7901 is substantially the sameas that in picture 7902. Accordingly, a motion vector is not detectedfrom these areas. On the other hand, the area occupied by image 7905showing a “circle” in picture 7901 is substantially different from thearea in picture 7902. Accordingly, a motion vector of the image 7905 isdetected from these areas.

The video encoder 7802 uses the detected motion vector to compress thepictures 7901 and 7902. On the other hand, the frame depth informationgeneration unit 7905 uses the motion vector VCT to calculate thebinocular parallax of the each image, such as the “house” image 7904 and“circle” image 7905. The video encoder 7802 further calculates the depthof each image from the image's binocular parallax. The informationindicating the depth of each image may be organized into a matrix 7906the same size as the matrix of the macroblocks in pictures 7901 and7902, as shown in FIG. 79C. In this matrix 7906, blocks 7907 are inone-to-one correspondence with the macroblocks 7903 in pictures 7901 and7902. Each block 7907 indicates the depth of the image shown by thecorresponding macroblocks 7903 by using, for example, a depth of 8 bits.In the example shown in FIG. 79, the depth of the image 7905 of the“circle” is stored in each of the blocks in an area 7908 in the matrix7906. This area 7908 corresponds to the entire areas in the pictures7901 and 7902 that represent the image 7905.

The video encoder 7802 may use the depth information to generate a depthmap for the left view or right view. In this case, the video encoder7802 respectively encodes either the left-view or right-view stream dataand the corresponding depth map stream as the base-view video stream andthe depth map stream by using the predictive encoding between picturesincluded in the video encoder 7802 itself. Each video stream 7812 afterthe conversion is stored in the database unit 7801.

The video encoder 7802 may further use the depth information tocalculate width WDH of the strip AL or AR in the vertical direction thatis included in either of the left view LV and the right view RV shown inFIGS. 60B and 60C and height HGT of the strip AT or AB in the horizontaldirection that is included in either of the left view LV and the rightview RV shown in FIGS. 61B and 61C. Actually, when an image of an objectis included in a vertical or horizontal strip, the motion vector of thisimage is detected as indicating “frame out” from the left view to theright view or vice-versa. Accordingly, the video encoder 7802 cancalculate the width or height of each strip from this motion vector.Information 7811 indicating the calculated width and height (hereinafterreferred to as “mask area information”) is stored in the database unit7801.

When encoding a secondary video stream from 2D video image data, thevideo encoder 7802 may also create offset information 7810 for asecondary video plane in accordance with operations of the authoringstaff. The generated offset information 7810 is stored in the databaseunit 7801.

The material creation unit 7803 creates elementary streams other thanvideo streams, such as an audio stream 7813, PG stream 7814, and IGstream 7815 and stores the created streams into the database unit 7801.For example, the material creation unit 7803 receives uncompressed LPCMaudio data from the authoring staff, encodes the uncompressed LPCM audiodata in accordance with a compression encoding method such as AC-3, andconverts the encoded LPCM audio data into the audio stream 7813. Thematerial creation unit 7803 additionally receives a subtitle informationfile from the authoring staff and creates the PG stream 7814 inaccordance with the subtitle information file. The subtitle informationfile defines image data or text data for showing subtitles, displaytimings of the subtitles, and visual effects to be added to thesubtitles, such as fade-in and fade-out. Furthermore, the materialcreation unit 7803 receives bit map data and a menu file from theauthoring staff and creates the IG stream 7815 in accordance with thebit map data and the menu file. The bit map data shows images that areto be displayed on a menu. The menu file defines how each button on themenu is to be transitioned from one status to another and defines visualeffects to be added to each button.

In response to operations by the authoring staff, the material creationunit 7803 furthermore creates offset information 7810 corresponding tothe PG stream 7814 and IG stream 7815. In this case, the materialcreation unit 7803 may use the depth information DPI generated by thevideo encoder 7802 to adjust the depth of the 3D graphics video imageswith the depth of the 3D video images. In this case, when the depth ofthe 3D video images changes greatly per each frame, the materialcreation unit 7803 may further process a series of offset values createdwith use of the depth information DPI in the low-path filter to decreasethe change per each frame. The offset information 7810 created in thisway is stored in the database unit 7801.

The scenario generation unit 7804 creates BD-ROM scenario data 7817 inresponse to an instruction received from the authoring staff via GUI andthen stores the created BD-ROM scenario data 7817 in the database unit7801. The BD-ROM scenario data 7817 defines methods of playing back theelementary streams 7812-7816 stored in the database unit 7801. Of thefile group shown in FIG. 2, the BD-ROM scenario data 7817 includes theindex file 211, the movie object file 212, and the playlist files221-223. The scenario generation unit 7804 further creates a parameterfile PRF and transfers the created parameter file PRF to the multiplexprocessing unit 7806. The parameter file PRF defines, from among theelementary streams 7812-7816 stored in the database unit 7801, streamdata to be multiplexed into the main TS and sub-TS.

The BD program creation unit 7805 provides the authoring staff with aprogramming environment for programming BD-J objects and Javaapplication programs. The BD program creation unit 7805 receives arequest from a user via GUI and creates each program's source codeaccording to the request. The BD program creation unit 7805 furthercreates a BD-J object file 251 from the BD-J objects and compresses theJava application programs in the JAR file 261. The program files BDP aretransferred to the format processing unit 7807.

In this context, it is assumed that a BD-J object is programmed in thefollowing way: the BD-J object causes the program execution unit 4234shown in FIG. 42 to transfer graphics data for GUI to the system targetdecoder 4225. Furthermore, the BD-J object causes the system targetdecoder 4225 to process graphics data as image plane data and to outputimage plane data to the plane adder 4226 in 1 plane+offset mode. In thiscase, the BD program creation unit 7805 may create offset information7810 corresponding to the image plane and store the offset information7810 in the database unit 7801. Here, the BD program creation unit 7305may use the depth information DPI generated by the video encoder 7802when creating the offset information 7810.

In accordance with the parameter file PRF, the multiplex processing unit7806 multiplexes each of the elementary streams 7812-7816 stored in thedatabase unit 7801 to form a stream file in MPEG-2 TS format. Morespecifically, as shown in FIG. 4, first each of the elementary streams7812-7815 is converted into a source packet sequence, and the sourcepackets included in each sequence are assembled to construct a singlepiece of multiplexed stream data. In this way, the main TS and sub-TSare created. These pieces of multiplexed stream data MSD are output tothe format processing unit 7807.

Furthermore, the multiplex processing unit 7806 creates the offsetmetadata based on the offset information 7810 stored in the databaseunit 7801. As shown in FIG. 11, the created offset metadata 1110 isstored in the dependent-view video stream. At this point, the mask areainformation 7811 stored in the database unit 7801 is stored in thedependent-view video stream together with the offset metadata. Note thatthe multiplex processing unit 7806 may process each piece of graphicsdata to adjust the arrangement of the graphics elements in the left andright video image frames so that the 3D graphics images represented byeach graphics plane are not displayed as overlapping in the same visualdirection as 3D graphics images represented by the other graphicsplanes. Alternatively, the multiplex processing unit 7806 may adjust theoffset value for each graphics plane so that the depths of 3D graphicsimages do not overlap.

Additionally, the multiplex processing unit 7806 creates a 2D clipinformation file and a dependent-view clip information file via thefollowing four steps. (I) Create entry maps 2230 shown in FIG. 23 forthe file 2D and file DEP. (II) Create the extent start points 2242 and2420 shown in FIGS. 24A and 24B using each file's entry map. At thispoint, it aligns extent ATC times between consecutive data blocks (seebelow). Furthermore, it designs the arrangement of extents so that thesizes of 2D extents, base-view extents, dependent-view extents, andextents SS satisfy predetermined conditions (see the <<SupplementaryExplanation>> regarding these conditions). (III) Extract the streamattribute information 2220 shown in FIG. 22 from each elementary streamto be multiplexed into the main TS and sub-TS. (IV) Associate, as shownin FIG. 22, a combination of an entry map 2230, 3D meta data 2240, andstream attribute information 2220 with a piece of clip information 2210.Each clip information file CLI is thus created and transmitted to theformat processing unit 7307.

The format processing unit 7807 creates a BD-ROM disc image 7820 of thedirectory structure shown in FIG. 2 from (i) the BD-ROM scenario data7817 stored in the database unit 7801, (ii) a group of program files BDPsuch as BD-J object files created by the BD program creation unit 7805,and (iii) multiplexed stream data MSD and clip information files CLIgenerated by the multiplex processing unit 7806. In this directorystructure, UDF is used as the file system.

When creating file entries for each of the files 2D, files DEP, andfiles SS, the format processing unit 7807 refers to the entry maps and3D metadata included in the 2D clip information files and dependent-viewclip information files. The SPN for each entry point and extent startpoint is thereby used in creating each allocation descriptor. Inparticular, the value of the LBN and the extent size to be representedby each allocation descriptor are determined so as to express aninterleaved arrangement like the one shown in FIG. 19. As a result, eachbase-view data block is shared by a file SS and file 2D, and eachdependent-view data block is shared by a file SS and file DEP.

<Recording Method of BD-ROM Disc Image>

FIG. 80 is a flowchart of a method for recording movie content on aBD-ROM disc using the recording device 7800 shown in FIG. 78. Thismethod begins, for example, when power to the recording device 7800 isturned on.

In step S8001, the elementary streams, programs, and scenario data to berecorded on a BD-ROM disc are created. In other words, the video encoder7802 creates a video stream 7812. The material creation unit 7803creates an audio stream 7813, PG stream 7814, and IG stream 7815. Thescenario generation unit 7804 creates BD-ROM scenario data 7817. Thesecreated pieces of data 7812-7817 are stored in the database unit 7801.On the other hand, the video encoder 7802 creates offset information7810 and mask area information 7811 and stores these pieces ofinformation in the database unit 7801. The material creation unit 7803creates offset information 7810 and stores this information in thedatabase unit 7801. The scenario generation unit 7804 creates aparameter file PRF and transfers this file to the multiplex processingunit 7806. The BD program creation unit 7805 creates a group of programfiles BDP, which include a BD-J object file and a JAR file, andtransfers this group BDP to the format processing unit 7807. The BDprogram creation unit 7805 also creates offset information 7810 andstores this information in the database unit 7801. Thereafter,processing proceeds to step S8002.

In step S8002, the multiplex processing unit 7806 creates offsetmetadata based on the offset information 7810 stored in the databaseunit 7801. The created offset metadata is stored in the dependent-viewvideo stream along with the mask area information 7811. Thereafter,processing proceeds to step S8003.

In step S8003, the multiplex processing unit 7806 reads the elementarystreams 7812-7816 from the database unit 7801 in accordance with theparameter file PRF and multiplexes these streams into a stream file inMPEG2-TS format. Thereafter, processing proceeds to step S8004.

In step S8004, the multiplex processing unit 7806 creates a 2D clipinformation file and a dependent-view clip information file. Inparticular, during creation of the entry map and extent start points,the extent ATC time is aligned between contiguous data blocks.Furthermore, the sizes of 2D extents, base-view extents, dependent-viewextents, and extents SS are set to satisfy predetermined conditions.Thereafter, processing proceeds to step S8005.

In step S8005, the format processing unit 7807 creates a BD-ROM discimage 7820 from the BD-ROM scenario data 7817, group of program filesBDP, multiplexed stream data MDS, and clip information file CLI.Thereafter, processing proceeds to step S8006.

In step S8006, the BD-ROM disc image 7820 is converted into data forBD-ROM pressing. Furthermore, this data is recorded on a master BD-ROMdisc. Thereafter, processing proceeds to step S8007.

In step S8007, BD-ROM discs 101 are mass produced by pressing the masterobtained in step S8006. Processing thus concludes.

<Method to Align Extent ATC Times>

FIG. 81 is a schematic diagram showing a method to align extent ATCtimes between consecutive data blocks. First, ATSs along the same ATCtime axis are assigned to source packets stored in a base-view datablock (hereinafter, SP1) and source packets stored in a dependent-viewdata block (hereinafter, SP2). As shown in FIG. 81, the rectangles 8110and 8120 respectively represent SP1 #p (p=0, 1, 2, 3, . . . , k, k+1, .. . , i, i+1) and SP2 #q (q=0, 1, 2, 3, . . . , m, m+1, . . . , j).These rectangles 8110 and 8120 are arranged in order along the time axisby the ATS of each source packet. The position of the top of eachrectangle 8110 and 8120 represents the value of the ATS of the sourcepacket. The length AT1 of each rectangle 8110 and 8120 represents theamount of time needed for the 3D playback device to transfer one sourcepacket from the read buffer to the system target decoder.

From the ATS A1 of SP1 #0 until an extent ATC time T_(EXT) has passed,SP1, i.e. SP1 #0, 1, 2, . . . , k, is transferred from the read bufferto the system target decoder and stored as the (n+1)^(th) base-viewextent EXT1[n] in one base-view data block. Similarly, from the ATS A3of SP1 #(k+1) until an extent ATC time T_(EXT) has passed, SP1, i.e. SP1#(k+1), . . . , i, is transferred from the read buffer to the systemtarget decoder and stored as the (n+2)^(th) base-view extent EXT l [n+1]in the next base-view data block.

On the other hand, SP2, which is to be stored in as the (n+1)^(th)dependent-view extent EXT2[n] in one dependent-view data block, isselected as follows. First, the sum of the ATS A1 of SP1 #0 and theextent ATC time T_(EXT), A1+T_(EXT), is sought as ATS A3 of SP1 #(k+1)located at the top of the (n+2)^(th) base-view extent EXT1[n+1]. Next,SP2, i.e. SP2 #0, 1, 2, . . . , m, is selected. Transfer of SP2 from theread buffer to the system target decoder begins during the period fromATS A1 of SP1 #0 until ATS A3 of SP1 #(k+1). Accordingly, the top SP2,i.e. ATS A2 of SP2 #0, is always equal to or greater than the top SP1,i.e. ATS A1 of SP1 #0: A2≧A1. Furthermore, all of the ATS of SP2 #0-mare less than ATS A3 of SP1 #(k+1). In this context, completion oftransfer of the last SP2, i.e. SP #m, may be at or after ATS A3 of SP1#(k+1).

Similarly, SP2, which is to be stored as the (n+2)^(th) dependent-viewextent EXT2[n+1] in one dependent-view data block, is selected asfollows. First, ATS A5 of SP1 #(i+1) located at the top of the(n+3)^(th) base-view extent is sought as ATS A5=A3+T_(EXT). Next, SP2,i.e. SP2 #(m+1)−j, is selected. Transfer of SP2 from the read buffer tothe system target decoder begins during the period from ATS A3 of SP1#(k+1) until ATS A5 of SP1 #(i+1). Accordingly, the top SP2, i.e. ATS A4of SP2 #(m+1), is always equal to or greater than the top SP1, i.e. ATSA3 of SP1 #(k+1): A4≧A3. Furthermore, all of the ATS of SP2 #(m+1)−j areless than ATS A5 of SP1 #(k+1).

Embodiment 4

FIG. 110 is a functional block diagram of a playback device realized byusing the integrated circuit 3 according to Embodiment 4 of the presentinvention. This playback device reproduces data having the structuredescribed in previous embodiments.

A medium IF unit 1 receives or reads data from a medium and transmitsthe data to the integrated circuit 3. Note that the data includes datawith the structure described in previous embodiments. The medium IF unit1 is, for example, a disk drive if the medium is an optical disc or harddisk; a card IF if the medium is a semiconductor memory such as an SDcard, USB memory, etc.; a CAN tuner or Si tuner if the medium is abroadcast wave such as CATV or the like; and a network IF if the mediumis a network such as the Ethernet™, wireless LAN, wireless public line,etc.

A memory 2 temporarily stores both the data received or read from themedium and data being processed by the integrated circuit 3. ASynchronous Dynamic Random Access Memory (SDRAM),Double-Data-Rate×Synchronous Dynamic Random Access Memory (DDR×SDRAM;x=1, 2, 3, . . . ), etc., is used as the memory 2. Any number ofmemories 2 may be provided; as necessary, the memory 2 may be a singleelement or a plurality of elements.

An integrated circuit 3 is a system LSI that treats data transmittedfrom the medium IF unit 1 with video and audio processing. Theintegrated circuit 3 includes a main control unit 6, stream processingunit 5, signal processing unit 7, memory control unit 9, and AV outputunit 8.

The main control unit 6 includes a program memory and a processor core.The program memory pre-stores basic software such as the OS. Theprocessor core has a timer function and an interrupt function, andcontrols the entirety of the integrated circuit 3 in accordance withprograms stored, for example, in the program memory.

Under the control of the main control unit 6, the stream processing unit5 receives data transmitted from the medium via the medium IF unit 1,and then stores the data into the memory 2 via a data bus in theintegrated circuit 3, or separates the data into visual data and audiodata. As previously described, a 2D/left-view AV stream file includes aleft-view video stream and a right-view AV stream file includes aright-view video stream. Furthermore, the data on the medium consists ofthe 2D/left-view AV stream file and the right-view AV stream filedivided into a plurality of extents and alternately arranged extent byextent. Note that, of the data on the medium, the portions including theleft-view video stream are left-view data, and the portions includingthe right-view video stream are right-view data. When the integratedcircuit 3 receives left-view and right-view data, respectively, the maincontrol unit 6 controls the stream processing unit 5 to store the datainto a first area and a second area in the memory 2. Note that the firstand second areas in the memory 2 may be logically separated areas in asingle memory element, or physically different memory elements. Thefollowing explanation on Embodiment 4 assumes that the left-view dataand the right-view data are main-view data and sub-view data,respectively. A similar explanation may be true if the right-view dataand the left-view data are the main-view data and the sub-view data,respectively.

Under the control of the main control unit 6, the signal processing unit7 uses an appropriate method to decode visual data and audio dataseparated by the stream processing unit 5. The visual data is compressedwith an encoding method such as MPEG-2, MPEG-4 AVC, MPEG-4 MVC, SMPTEVC-1, etc. Audio data is compressed with an encoding method such asDolby AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD, linear PCM, etc. Thesignal processing unit 7 decodes the data with the corresponding method.Note that the model of the signal processing unit 7 may be equivalent toeither the combination of the TS priority filter and the variousdecoders shown in FIG. 45, or the various decoders shown in FIG. 46.Furthermore, the signal processing unit 7 extracts the metadata from theright-view video stream, and then notifies the AV output unit 8 of themetadata. Note that, as described above, the metadata is located at eachof GOPs constituting the right-view video stream, and includescombinations of offset information and offset identifiers.

When the model of the signal processing unit 7 is equivalent to thecombination of the TS priority filter and the various decoders shown inFIG. 45, the signal processing unit 7 first monitors the TS priorityflags of TS packets included in the right-view data, and then uses thevalues of the TS priority flags to select TS packets containing themetadata. The signal processing unit 7 next performs the following twoprocesses in parallel by using separate modules: a process of decodingthe TS packets containing picture data into uncompressed picture data,and a process of extracting the metadata from the TS packets containingthe metadata. When the model of the signal processing unit 7 isequivalent to the various decoders shown in FIG. 46, the signalprocessing unit 7 allows TS packets containing the right-view data to besent to the same decoder, regardless of the values of TS priority flags.The decoder performs both the two processes of decoding the TS packetsinto uncompressed picture data and extracting the metadata from the TSpackets. In this way, as far as data has the structure described inprevious embodiments, the signal processing unit 7 of either model cansuccessfully execute both the processes of decoding the data intouncompressed picture data and extracting metadata from the data.

The memory control unit 9 arbitrates accesses to the memory 2 by thefunction blocks in the integrated circuit 3.

Under the control of the main control unit 6, the AV output unit 8superimposes pieces of visual data decoded by the signal processing unit7, processes the pieces of visual data with format conversion or thelike, and then outputs them to the integrated circuit 3.

FIG. 111 is a functional block diagram showing a representativestructure of the stream processing unit 5. The stream processing unit 5is provided with a device stream IF unit 51, a demultiplexer 52, and aswitching unit 53.

The device stream IF unit 51 is an interface for data transfer betweenthe medium IF unit 1 and the integrated circuit 3. For example, thedevice stream IF unit 51 corresponds to a Serial Advanced TechnologyAttachment (SATA), Advanced Technology Attachment Packet Interface(ATAPI), or Parallel Advanced Technology Attachment (PATA) if the mediumis an optical disc or a hard disk; a card IF if the medium is asemiconductor memory such as an SD card, USB memory, etc.; a tuner IF ifthe medium is a broadcast wave such as CATV or the like; and a networkIF if the medium is a network such as the Ethernet™, a wireless LAN, ora wireless public line. Note that, depending on the type of medium, thedevice stream IF unit 51 may achieve part of the functions of the mediumIF unit 1, or the medium IF unit 1 may be embedded in the integratedcircuit 3 and used as the device stream IF unit 51.

The demultiplexer 52 separates visual data and audio data from playbackdata including video images and sounds; the playback data is included indata transmitted from the medium. Each of the above-described extentsconsists of source packets containing various data such as video images,sounds, PG (subtitles), and IG (menus). In some cases, however, sub-viewdata may not include an audio stream. Each extent is separated intovisual and audio TS packets depending on the PIDs (identifiers) includedin the source packets, and then transmitted to the signal processingunit 7. Processed data is transmitted to the signal processing unit 7directly or after temporary storage in the memory 2. Note that the modelof the demultiplexer 52 corresponds to the source depacketizers and thePID filters shown in FIG. 45, for example.

The switching unit 53 changes destinations for the output or storage ofdata that the device stream IF unit 51 receives. Specifically, when thedevice stream IF unit 51 receives left-view data, the switching unit 53switches the destination of the data to the first area of the memory 2,whereas when the device stream IF unit 51 receives right-view data, theswitching unit 53 switches the destination of the data to the secondarea of the memory 2. The switching unit 53 is, for example, a DirectMemory Access Controller (DMAC). FIG. 112 is a functional block diagramof the switching unit 53 and surrounding units when the switching unit53 is a DMAC. Under the control of the main control unit 6, the DMAC 53transmits received data and a destination address to the memory controlunit 9; the received data is data that the device stream IF unit 51 hasreceived, and the destination address is an address where the receiveddata is to be stored. Specifically, let's assume that addresses 1 and 2are the addresses of the first area and the second area in the memory 2,respectively. When the received data is left-view data, the DMAC 53transmits address 1 to the memory control unit 9, whereas when thereceived data is right-view data, the DMAC 53 transmits address 2. Thememory control unit 9 stores the received data into the memory 2 inaccordance with the destination addresses transmitted from the switchingunit 53. The destination for the output or storage of the received datais changed, depending on the type of the received data. Note that acircuit dedicated to the control over the switching unit 53 may beprovided, instead of the main control unit 6.

The device stream IF unit 51, demultiplexer 52, and switching unit 53were described as a representative structure of the stream processingunit 5. However, the stream processing unit 5 may be further providedwith an encryption engine, a security control unit, a controller fordirect memory access, or the like. The encryption engine receives anddecrypts encrypted data, key data, or the like. The security controlunit stores a private key and controls execution of a deviceauthentication protocol or the like between the medium and the playbackdevice. In the example described above, when data received from themedium is stored into the memory 2, the switching unit 53 changes thedestination for the storage of the data, depending on whether the datais left-view data or right-view data. Alternatively, after data receivedfrom the medium is temporarily stored in the memory 2, the data may beseparated into left-view data and right-view data while beingtransferred to the demultiplexer 52.

FIG. 113 is a functional block diagram showing a representativestructure of the AV output unit 8. The AV output unit 8 is provided withan image superimposition unit 81, video output format conversion unit82, and audio/video output IF unit 83.

The image superimposition unit 81 superimposes decoded visual data.Specifically, the image superimposition unit 81 superimposes PG(subtitles) data and IG (menus) data on the left-view video data andright-view video data picture by picture. The model of the imagesuperimposition unit 81 is shown, for example, in FIG. 47.

FIG. 114 is a schematic diagram showing one example of the method ofusing the memory 2 during the process of superimposing images. Thememory 2 includes a storage area of plane data corresponding to leftview, a storage area of plane data corresponding to right view, astorage area of plane data corresponding to graphics, and a storage areaof data of superimposed images. Each plane data storage area is an areawhere decoded data is temporarily stored before rendered in thecorresponding plane. The storage area of data of superimposed images isan area where data resulted from superimposing a graphics plane on aleft-view plane or a right-view plane is stored. Note that each planemay be an area in the memory 2 or a virtual space.

FIGS. 115 and 116 are schematic diagrams showing the processes ofsuperimposing images by using the memory 2 shown in FIG. 114. The imagesuperimposition unit 81 first provides an offset to a graphics planebased on the offset information included in the metadata extracted bythe signal processing unit 7. The image superimposition unit 81 thensuperimposes the graphics plane with the offset on a video plane.Specifically, FIG. 115 shows that the image superimposition unit 81provides an offset of +X to a graphics plane, and then superimposes thegraphics plane on a left-view plane. On the other hand, FIG. 116 showsthat the image superimposition unit 81 provides an offset of −X to theoriginal graphics plane, and then superimposes the graphics plane on aright-view plane. The value X is the offset value represented by anumber of pixels. These superimposition processes as shown in thefigures allow data pieces of pixels located at the same horizontalcoordinates to be combined with each other. The data after superimposedis stored in the storage area of data of superimposed images in thememory 2.

FIG. 117 is a schematic diagram showing another example of the method ofusing the memory 2 during the process of superimposing images. Thememory 2 further includes “storage areas of plane data corresponding tographics with offset (for superimposed left and right views)”. Thegraphics planes with offsets are to be temporarily stored into theseplane data storage areas before superimposed on left-view and right-viewplanes. For example, the image superimposition unit 81 provides anoffset of +X to the graphics plane, and temporarily stores the graphicsplane into the “the storage area of plane data corresponding to graphicswith offset (for superimposed left view)”. The image superimpositionunit 81 then reads the graphics plane from the plane data storage area,superimposes the graphics plane on the left-view plane, and stores theresult of superimposition into the storage area of data of superimposedimages. On the other hand, the image superimposition unit 81 provides anoffset −X to the graphics plane, and temporarily stores the plane datainto the “the storage area of plane data corresponding to graphics withoffset (for superimposed right view)”. The image superimposition unit 81then reads the graphics plane from the plane data storage area,superimposes the graphics plane on the right-view plane, and stores theresult of superimposition into the storage area of data of superimposedimages.

The video output format conversion unit 82 processes visual data afterdecoded or superimposed, as necessary, with resizing, IP conversion,noise reduction, frame rate conversion, or the like. Resizing is aprocess to enlarge or reduce the sizes of images. IP conversion is aprocess to convert between progressive scanning and interlaced scanningNoise reduction is a process to remove noise from images. Frame rateconversion is a process to change frame rates. The video output formatconversion unit 82 sends the data after processed to the imagesuperimposition unit 81 or the audio/video output IF unit 83.

The audio/video output IF unit 83 converts the visual data processed bythe video output format conversion unit 82 and the decoded audio datainto predetermined data transmission formats by an encoding process orthe like. Note that part of the audio/video output IF unit 83 may beprovided externally to the integrated circuit 3, as described below.

FIG. 118 is a detailed functional block diagram of the AV output unit 8and the data output unit in the playback device. The AV output unit 8and the data output unit in the playback device support a plurality ofdata transmission formats. Specifically, as shown in FIG. 118, theaudio/video output IF unit 83 includes an analog video output IF unit 83a, analog audio output IF unit 83 c, and digital video/audio output IFunit 83 b.

The analog video output IF unit 83 a converts the visual data processedby the video output format conversion unit 82 into an analog videosignal format, and then outputs the visual data. The analog video outputIF unit 83 a includes, for example, a composite video encoder supportingone of the NTSC, PAL, and SECAM formats, encoder for S-video signal (Y/Cseparation), encoder for component video signal, and D/A converter(DAC).

The digital audio/video output IF unit 83 b merges the decoded audiodata and the visual data processed by the video output format conversionunit 82, and further encrypts the merged data. After that, the digitalaudio/video output IF unit 83 b encodes and outputs the encrypted datain accordance with data transmission standards. For example, the HDMIcommunication unit shown in FIG. 42 corresponds to the digitalvideo/audio output IF unit 83 b.

The analog audio output IF unit 83 c processes the decoded audio datawith D/A conversion, and then outputs analog audio data. An audio DAC orthe like corresponds to the analog audio output IF unit 83 c.

The AV output unit 8 and the data output unit in the playback device canchange the transmission formats of the visual data and audio data,depending on the data reception devices or data input terminals that thedisplay device and speaker 4 supports. The AV output unit 8 and the dataoutput unit in the playback device can also allow an user to select thetransmission formats thereof. Furthermore, they can transmit dataconverted from the same content in not only a single transmissionformat, but also two or more transmission formats in parallel.

The image superimposition unit 81, video output format conversion unit82, and audio/video output IF unit 83 were described as a representativestructure of the AV output unit 8. However, the AV output unit 8 may befurther provided with a graphics engine and the like. The graphicsengine treats data decoded by the signal processing unit 7 with graphicsprocessing such as filtering, combining screens, drawing curves, and 3Ddisplay.

This concludes the description on the structure of the playback deviceaccording to Embodiment 4. Note that all of the above-described functionblocks need not be built in the integrated circuit 3. Conversely, thememory 2 may be built in the integrated circuit 3. The above descriptionon Embodiment 4 explains the main control unit 6 and signal processingunit 7 as separate function blocks. However, the main control unit 6 mayperform part of the processing that the signal processing unit 7 shouldperform.

The topology of a control bus and a data bus connecting between thefunction blocks in the integrated circuit 3 may be selected to suit theprocedure and details of processing by the function blocks. FIGS. 119Aand 119B are schematic diagrams showing examples of the topology of thecontrol bus and data bus in the integrated circuit 3. As shown in FIG.119A, both the control bus 21 and the data bus 22 are arranged so thateach of the function blocks 5-9 is directly connected with all the otherfunction blocks. Alternatively, as shown in FIG. 119B, the data bus 23is arranged so that each of the function blocks 5-8 is directlyconnected only with the memory control unit 9. In this case, each of thefunction blocks 5-8 transfers data to the other function blocks via thememory control unit 9 and further the memory 2.

The integrated circuit 3 may be a multi-chip module, instead of an LSIimplemented on a single chip. In that case, a plurality of chipsconstituting the integrated circuit 3 are sealed in a single package,and thus the integrated circuit 3 looks like a single LSI.Alternatively, the integrated circuit 3 may be configured by using aField Programmable Gate Array (FPGA) or a reconfigurable processor. TheFPGA is an LSI programmable after manufactured. The reconfigurableprocessor is an LSI that allows connections between internal circuitcells and settings for each circuit cell to be reconfigured.

<Incorporation of Integrated Circuit 3 into Display Device>

An integrated circuit similar to the above-described integrated circuit3 may be incorporated in a display device to cause the display device toperform the above-described processing by the playback device accordingto Embodiment 4. FIG. 120 is a functional block diagram showing thestructure of the integrated circuit incorporated in the display deviceand the surrounding units thereof. As shown in FIG. 120, the integratedcircuit 3 uses a medium IF unit 1 and a memory 2 to treat the datareceived by the medium IF unit 1 with processing similar to theabove-described signal processing. The visual data processed by theintegrated circuit 3 is sent to the display driving unit 10. The displaydriving unit 10 controls the display panel 11 in accordance with thevisual data. As a result, the visual data is outputted as images on thescreen of the display panel 11. On the other hand, the audio dataprocessed by the integrated circuit 3 is outputted as sounds via thespeaker 12.

FIG. 121 is a detailed functional block diagram of the AV output unit 8shown in FIG. 120. The AV output unit 8 includes a video output IF unit84 and an audio output IF unit 85, in contrast to that shown in FIG.118. The video output IF unit 84 and the audio output IF unit 85 may beprovided inside or outside the integrated circuit 3. The video output IFunit 84 transfers visual data from the video output format conversionunit 82 to the display driving unit 10. The audio output IF unit 85transfers audio data from the signal processing unit 7 to the speaker12. Note that two or more of units each similar to the video output IFunit 84 or the audio output IF unit 85 may be provided. In addition, thevideo output IF unit 84 and the audio output IF unit 85 may beintegrated in one unit.

<Playback Processing by Playback Device Using Integrated Circuit 3>

FIG. 122 is a flowchart of playback processing by the playback deviceusing the integrated circuit 3. The playback processing starts when themedium IF unit 1 is connected with a medium to be able to receive datatherefrom, for example, when an optical disc is inserted into the discdrive. During the playback processing, the playback device receives anddecodes data from the medium. The playback device then outputs thedecoded data as a video signal and an audio signal.

In step S1, the medium IF unit 1 receives or reads data from the mediumand transfers the data to the stream processing unit 5. Processing thenproceeds to step S2.

In step S2, the stream processing unit 5 separates visual data and audiodata from the data received or read in step S1. Processing then proceedsto step S3.

In step S3, the signal processing unit 7 decodes each type of dataseparated in step S2 by a method appropriate for the method of encodingthe data. In parallel with the decoding, the signal processing unit 7further extracts metadata from right-view data, and notifies the AVoutput unit 8 of the metadata. Note that the signal processing unit 7may monitor the TS priority flags of the TS packets included in theright-view data to select TS packets containing the metadata.Alternatively, the signal processing unit 7 may cause the same decoderto both decode the TS packets into uncompressed picture data and extractthe metadata from the TS packets. Processing then proceeds to step S4.

In step S4, the AV output unit 8 superimposes the visual data decoded instep S3. As necessary, the AV output unit 8 retrieves and uses offsetinformation from the metadata extracted in step S3. Processing thenproceeds to step S5.

In step S5, the AV output unit 8 outputs the visual data and audio dataprocessed in steps S2-S4. Processing then proceeds to step S6.

In step S6, the main control unit 6 determines whether or not tocontinue the playback processing. Processing again proceeds from stepS1, for example, when data to be received or read by the medium IF unit1 remains in the medium. On the other hand, processing ends when themedium IF unit 1 finishes receiving or reading data from the mediumbecause, for example, the optical disc has been removed from the discdrive or the user has instructed to stop playback.

FIG. 123 is a flowchart showing details of the steps S1-S6 shown in FIG.122. Steps S101-S110 shown in FIG. 123 are performed under control ofthe main control unit 6. Steps S101-S103 mainly correspond to details ofstep S1, steps S104 to details of step S2, step S105 to details of stepS3, steps S106-S108 to details of step S4, and steps S109 and S110 todetails of step S5.

In step S101, before receiving or reading data to be played back fromthe medium via the medium IF unit 1, the device stream IF unit 51receives or reads data required for the playback processing such as aplaylist file, clip information file, and the like, from the medium viathe medium IF unit 1. The device stream IF unit 51 further stores thedata required into the memory 2 via the memory control unit 9.Processing then proceeds to step S102.

In step S102, the main control unit 6 identifies the encoding formats ofvideo data and audio data stored in the medium, based on the streamattribute information included in the clip information file. The maincontrol unit 6 further initializes the signal processing unit 7 to beable to perform decoding in a manner corresponding to the identifiedencoding format. Processing then proceeds to step S103.

In step S103, the device stream IF unit 51 receives or reads video andaudio data to be played back from the medium via the medium IF unit 1.In particular, the data is received or read extent by extent. The devicestream IF unit 51 further stores the data into the memory 2 via theswitching unit 53 and memory control unit 9. In particular, whenleft-view data is received or read, the main control unit 6 controls theswitching unit 53 to switch the destination for storage of the left-viewdata to the first area in the memory 2. On the other hand, whenright-view data is received or read, the main control unit 6 controlsthe switching unit 53 to switch the destination for storage of theright-view data to the second area in the memory 2. Processing thenproceeds to step S104.

In step S104, the data stored in the memory 2 is transferred to thedemultiplexer 52 in the stream processing unit 5. The demultiplexer 52first reads PIDs from source packets constituting the data. Thedemultiplexer 52 then uses the PIDs to distinguish whether the TSpackets included in the source packets are visual data or audio data.The demultiplexer 52 further transmits each TS packet to a correspondingdecoder in the signal processing unit 7 depending on the result ofdistinguishing. Note that the signal processing unit 7 may monitor theTS priority flags of the TS packets included in the right-view data tosend the TS packets containing the metadata to a dedicated moduleseparate from the primary video decoder, i.e., the offset metadataprocessing unit. Processing then proceeds to step S105.

In step S105, each decoder in the signal processing unit 7 decodes thetransmitted TS packets with an appropriate method. In parallel with thedecoding, the signal processing unit 7 further extracts the metadatafrom the right-view data and notifies the AV output unit 8 of themetadata. Note that the extracting may be performed by the offsetmetadata processing unit separate from the primary video decoder, or maybe performed by the primary video decoder simultaneously with thedecoding. Processing then proceeds to step S106.

In step S106, pictures of the left-view video stream and right-viewvideo stream decoded by the signal processing unit 7 are sent to thevideo output format conversion unit 82. The video output formatconversion unit 82 resizes the pictures to match the resolution of thedisplay device 4. Processing then proceeds to step S107.

In step S107, the image superimposition unit 81 receives video planedata composed of the pictures resized in step S106 from the video outputformat conversion unit 82. The image superimposition unit 81 alsoreceives decoded PG and IG plane data from the signal processing unit 7.The image superimposition unit 81 further superimposes the decoded PGand IG plane data. Processing then proceeds to step S108.

In step S108, the video output format conversion unit 82 receives planedata superimposed in step S107 from the image superimposition unit 81.The video output format conversion unit 82 further processes the planedata with IP conversion. Processing then proceeds to step S109.

In step S109, the audio/video output IF unit 83 receives the visual dataprocessed with the IP conversion in step S108 from the video outputformat conversion unit 82, and receives decoded audio data from thesignal processing unit 7. The audio/video output IF unit 83 furtherprocesses the visual and audio data with encoding, D/A conversion, andthe like, in accordance with the method of outputting data by thedisplay device and speaker 4 and the method of transmitting data to thedisplay device and speaker 4. With these processes, the visual data andaudio data are each converted into an analog output format or a digitaloutput format. Visual data in the analog output format includes acomposite video signal, S-video signal, component video signal, and thelike. Also, HDMI or the like is supported as the digital output format.Processing then proceeds to step S110.

In step S110, the audio/video output IF unit 83 transmits the visualdata and audio data processed in step S109 to the display device andspeaker 4. Processing then proceeds to step S6. Details of step S6 canbe found in the above description.

In each of the above-described steps, each time data is processed, thedata may be temporarily stored into the memory 2. Resizing and IPconversion by the video output format conversion unit 82 in steps S106and S108 may be skipped, if not necessary. Instead of or in addition tothese processes, other processes such as noise reduction and frame rateconversion may be further performed. In addition, the order of theprocesses may be changed insofar as possible.

When the display device shown in FIG. 120 is used in playbackprocessing, the flowchart of the playback processing is basicallysimilar to the flowchart shown in FIGS. 122 and 123. The function blocksshown in FIGS. 120 and 121 operate in a similar manner to those shown inFIGS. 110 and 113.

Supplementary Explanation

<Principle of 3D Video Image Playback>

Playback methods of 3D video images are roughly classified into twocategories: methods using a holographic technique, and methods usingparallax video.

A method using a holographic technique is characterized by allowing theviewer to perceive objects in video as stereoscopic by giving theviewer's visual perception substantially the same information as opticalinformation provided to visual perception by human beings of actualobjects. A technical theory for utilizing these methods for moving videodisplay has been established. However, it is extremely difficult toconstruct, with present technology, a computer that is capable ofreal-time processing of the enormous amount of calculation required formoving video display and a display device having super-high resolutionof several thousand lines per 1 mm. Accordingly, at the present time,the realization of these methods for commercial use is hardly in sight.

“Parallax video” refers to a pair of 2D video images shown to each ofthe viewer's eyes for the same scene, i.e. the pair of a left view and aright view. A method using parallax video is characterized by playingback the left-view and right-view of a single scene so that the viewersees each view in only one eye, thereby allowing the user to perceivethe scene as stereoscopic.

FIGS. 82A, 82B, and 82C are schematic diagrams illustrating theprinciple behind playback of 3D video images (stereoscopic video images)in a method using parallax video images. FIG. 82A is a top view of theviewer VWR looking at a cube CBC placed directly in front of theviewer's face. FIGS. 82B and 82C are schematic diagrams showing theouter appearance of the cube CBC as a 2D video image as perceivedrespectively by the left eye LEY and the right eye REY of the viewerVWR. As is clear from comparing FIG. 82B and FIG. 82C, the outerappearances of the cube CBC as perceived by the eyes are slightlydifferent. The difference in the outer appearances, i.e., the binocularparallax allows the viewer VWR to recognize the cube CBC asthree-dimensional. Thus, according to a method using parallax video,left and right 2D video images with different viewpoints are firstprepared for a single scene. For example, for the cube CBC shown in FIG.82A, the left view of the cube CBC shown in FIG. 96B and the right viewshown in FIG. 82C are prepared. In this context, the position of eachviewpoint is determined by the binocular parallax of the viewer VWR.Next, each 2D video image is played back so as to be perceived only bythe corresponding eye of the viewer VWR. Consequently, the viewer VWRrecognizes the scene played back on the screen, i.e., the video image ofthe cube CBC, as stereoscopic. Unlike methods using a holographytechnique, methods using parallax video thus have the advantage ofrequiring preparation of 2D video images from merely two viewpoints.

Several concrete methods for how to use parallax video have beenproposed. From the standpoint of how these methods show left and right2D video images to the viewer's eyes, the methods are divided intoalternate frame sequencing methods, methods that use a lenticular lens,two-color separation methods, etc.

In the alternate frame sequencing method, left and right 2D video imagesare alternately displayed on a screen for a predetermined time, whilethe viewer watches the screen using shutter glasses. Each lens in theshutter glasses is formed by a liquid crystal panel, for example. Thelenses pass or block light in a uniform and alternate manner insynchronization with switching of the 2D video images on the screen.That is, each lens functions as a shutter that periodically blocks aneye of the viewer. More specifically, while a left-video image isdisplayed on the screen, the shutter glasses make the left-side lenstransmit light and the right-hand side lens block light. Conversely,while a right-video image is displayed on the screen, the shutterglasses make the right-side lens transmit light and the left-side lensblock light. As a result, the viewer sees afterimages of the right andleft-video images overlaid on each other and thus perceives a single 3Dvideo image.

According to the alternate-frame sequencing method, as described above,right and left-video images are alternately displayed in a predeterminedcycle. For example, when 24 video frames are displayed per second forplaying back normal 2D video images, 48 video frames in total for bothright and left eyes need to be displayed for 3D video images.Accordingly, a display device capable of quickly executing rewriting ofthe screen is preferred for this method.

In a method using a lenticular lens, a right-video frame and aleft-video frame are respectively divided into vertically long andnarrow rectangular shaped small areas. The small areas of theright-video frame and the small areas of the left-video frame arealternately arranged in a horizontal direction on the screen anddisplayed at the same time. The surface of the screen is covered by alenticular lens. The lenticular lens is a sheet-shaped lens constitutedfrom multiple long and thin hog-backed lenses arranged in parallel. Eachhog-backed lens lies in the longitudinal direction on the surface of thescreen. When the viewer sees the left and right-video frames through thelenticular lens, only the viewer's left eye perceives light from thedisplay areas of the left-video frame, and only the viewer's right eyeperceives light from the display areas of the right-video frame. Theviewer thus sees a 3D video image from the binocular parallax betweenthe video images respectively perceived by the left and right eyes. Notethat according to this method, another optical component having similarfunctions, such as a liquid crystal device, may be used instead of thelenticular lens. Alternatively, for example, a longitudinal polarizationfilter may be provided in the display areas of the left image frame, anda lateral polarization filter may be provided in the display areas ofthe right image frame. In this case, the viewer sees the screen throughpolarization glasses. In the polarization glasses, a longitudinalpolarization filter is provided for the left lens, and a lateralpolarization filter is provided for the right lens. Consequently, theright and left-video images are each perceived only by the correspondingeye, thereby allowing the viewer to perceive 3D video images.

In a method using parallax video, in addition to being constructed fromthe start by a combination of left and right-video images, the 3D videocontent can also be constructed from a combination of 2D video imagesand a depth map. The 2D video images represent 3D video images projectedon a hypothetical 2D screen, and the depth map represents the depth ofeach pixel in each portion of the 3D video images as compared to the 2Dscreen. When the 3D content is constructed from a combination of 2Dvideo images with a depth map, the 3D playback device or display devicefirst constructs left and right-video images from the combination of 2Dvideo images with a depth map and then creates 3D video images fromthese left and right-video images using one of the above-describedmethods.

FIG. 83 is a schematic diagram showing an example of constructing aleft-view LVW and a right-view RVW from the combination of a 2D videoimage MVW and a depth map DPH. As shown in FIG. 83, a circular plate DSCis shown in the background BGV of the 2D video image MVW. The depth mapDPH indicates the depth for each pixel in each portion of the 2D videoimage MVW. According to the depth map DPH, in the 2D video image MVW,the display area DA1 of the circular plate DSC is closer to the viewerthan the screen, and the display area DA2 of the background BGV isdeeper than the screen. The parallax video generation unit PDG in theplayback device first calculates the binocular parallax for each portionof the 2D video image MVW using the depth of each portion indicated bythe depth map DPH. Next, the parallax video generation unit PDG shiftsthe presentation position of each portion in the 2D video image MVW tothe left or right in accordance with the calculated binocular parallaxto construct the left-view LVW and the right-view RVW. In the exampleshown in FIG. 83, the parallax video generation unit PDG shifts thepresentation position of the circular plate DSC in the 2D video imageMVW as follows: the presentation position of the circular plate DSL inthe left-view LVW is shifted to the right by half of its binocularparallax, S1, and the presentation position of the circular plate DSR inthe right-view RVW is shifted to the left by half of its binocularparallax, S1. In this way, the viewer perceives the circular plate DSCas being closer than the screen. Conversely, the parallax videogeneration unit PDG shifts the presentation position of the backgroundBGV in the 2D video image MVW as follows: the presentation position ofthe background BGL in the left-view LVW is shifted to the left by halfof its binocular parallax, S2, and the presentation position of thebackground BGR in the right-view RVW is shifted to the right by half ofits binocular parallax, S2. In this way, the viewer perceives thebackground BGV as being deeper than the screen.

A playback system for 3D video images with use of parallax video is ingeneral use, having already been established for use in movie theaters,attractions in amusement parks, and the like. Accordingly, this methodis also useful for implementing home theater systems that can play back3D video images. In the embodiments of the present invention, amongmethods using parallax video, an alternate-frame sequencing method or amethod using polarization glasses is assumed to be used. However, apartfrom these methods, the present invention can also be applied to other,different methods, as long as they use parallax video. This will beobvious to those skilled in the art from the above explanation of theembodiments.

<File System on the BD-ROM Disc>

When UDF is used as the file system for the BD-ROM disc 101, the volumearea 202B shown in FIG. 2 generally includes areas in which a pluralityof directories, a file set descriptor, and a terminating descriptor arerespectively recorded. Each “directory” is a data group composing thedirectory. A “file set descriptor” indicates the LBN of the sector inwhich a file entry for the root directory is stored. The “terminatingdescriptor” indicates the end of the recording area for the file setdescriptor.

Each directory shares a common data structure. In particular, eachdirectory includes a file entry, directory file, and a subordinate filegroup.

The “file entry” includes a descriptor tag, Information Control Block(ICB) tag, and allocation descriptor. The “descriptor tag” indicatesthat the type of the data that includes the descriptor tag is a fileentry. For example, when the value of the descriptor tag is “261”, thetype of that data is a file entry. The “ICB tag” indicates attributeinformation for the file entry itself. The “allocation descriptor”indicates the LBN of the sector on which the directory file belonging tothe same directory is recorded.

The “directory file” typically includes a plurality of each of a fileidentifier descriptor for a subordinate directory and a file identifierdescriptor for a subordinate file. The “file identifier descriptor for asubordinate directory” is information for accessing the subordinatedirectory located directly below that directory. This file identifierdescriptor includes identification information for the subordinatedirectory, directory name length, file entry address, and actualdirectory name. In particular, the file entry address indicates the LBNof the sector on which the file entry of the subordinate directory isrecorded. The “file identifier descriptor for a subordinate file” isinformation for accessing the subordinate file located directly belowthat directory. This file identifier descriptor includes identificationinformation for the subordinate file, file name length, file entryaddress, and actual file name. In particular, the file entry addressindicates the LBN of the sector on which the file entry of thesubordinate file is recorded. The “file entry of the subordinate file”,as described below, includes address information for the dataconstituting the actual subordinate file.

By tracing the file set descriptors and the file identifier descriptorsof subordinate directories/files in order, the file entry of anarbitrary directory/file recorded on the volume area 202B can beaccessed. Specifically, the file entry of the root directory is firstspecified from the file set descriptor, and the directory file for theroot directory is specified from the allocation descriptor in this fileentry. Next, the file identifier descriptor for the directoryimmediately below the root directory is detected from the directoryfile, and the file entry for that directory is specified from the fileentry address therein. Furthermore, the directory file for thatdirectory is specified from the allocation descriptor in the file entry.Subsequently, from within the directory file, the file entry for thesubordinate directory or subordinate file is specified from the fileentry address in the file identifier descriptor for that subordinatedirectory or subordinate file.

“Subordinate files” include extents and file entries. The “extents” area generally multiple in number and are data sequences whose logicaladdresses, i.e. LBNs, are consecutive on the disc. The entirety of theextents comprises the actual subordinate file. The “file entry” includesa descriptor tag, ICB tag, and allocation descriptors. The “descriptortag” indicates that the type of the data that includes the descriptortag is a file entry. The “ICB tag” indicates attribute information forthe file entry itself. The “allocation descriptors” are provided in aone-to-one correspondence with each extent and indicate the arrangementof each extent on the volume area 202B, specifically the size of eachextent and the LBN for the top of the extent. Accordingly, by referringto each allocation descriptor, each extent can be accessed. Also, thetwo most significant bits of each allocation descriptor indicate whetheran extent is actually recorded on the sector for the LBN indicated bythe allocation descriptor. Specifically, when the two most significantbits are “0”, an extent has been assigned to the sector and has beenactually recorded thereat. When the two most significant bits are “1”,an extent has been assigned to the sector but has not been yet recordedthereat.

Like the above-described file system employing a UDF, when each filerecorded on the volume area 202B is divided into a plurality of extents,the file system for the volume area 202B also generally stores theinformation showing the locations of the extents, as with theabove-mentioned allocation descriptors, in the volume area 202B. Byreferring to the information, the location of each extent, particularlythe logical address thereof, can be found.

<Size of Data Blocks and Extent Blocks>

As shown in FIG. 19, the multiplexed stream data on the BD-ROM disc 101is arranged by being divided into dependent-view data blocks D[n] andbase-view data blocks B[n] (n=0, 1, 2, 3, . . . ). Furthermore, thesedata block groups D[n] and B[n] are recorded consecutively on a track inan interleaved arrangement to form a plurality of extent blocks1901-1903. To ensure seamless playback of both 2D video images and 3Dvideo images from these extent blocks 1901-1903, the size of each datablock and each extent block 1901-1903 should meet the followingconditions based on the capability of the playback device 102.

<<Conditions Based on Capability in 2D Playback Mode>>

FIG. 84 is a block diagram showing playback processing in the playbackdevice 102 in 2D playback mode. As shown in FIG. 84, this playbackprocessing system includes the BD-ROM drive 3701, read buffer 3721, andsystem target decoder 4225 shown in FIG. 37. The BD-ROM drive 3701 reads2D extents from the BD-ROM disc 101 and transfers the 2D extents to theread buffer 3721 at a read rate R_(UD54). The system target decoder 4225reads source packets from each 2D extent stored in the read buffer 3721at a mean transfer rate R_(EXT2D) and decodes the source packets intovideo data VD and audio data AD.

The mean transfer rate R_(EXT2D) equals 192/188 times the mean rate ofprocessing by the system target decoder 4225 to extract TS packets fromeach source packet. In general, this mean transfer rate R_(EXT2D)changes for each 2D extent. The maximum value R_(MAX2D) of the meantransfer rate R_(EXT2D) equals 192/188 times the system rate R_(TS) forthe file 2D. In this case, the coefficient 192/188 is the ratio of bytesin a source packet to bytes in a TS packet. The mean transfer rateR_(EXT2D) is conventionally represented in bits/second and specificallyequals the value of the size of a 2D extent expressed in bits divided bythe extent ATC time. The “size of an extent expressed in bits” is eighttimes the product of the number of source packets in the extent and thenumber of bytes per source packet (=192 bytes×8 bits/byte).

The read rate R_(UD54) is conventionally expressed in bits/second and isset at a higher value, e.g. 54 Mbps, than the maximum value R_(MAX2D) ofthe mean transfer rate R_(EXT2D):R_(UD54)>R_(MAX2D). This preventsunderflow in the read buffer 3721 due to decoding processing by thesystem target decoder 4225 while the BD-ROM drive 3701 is reading a 2Dextent from the BD-ROM disc 101.

FIG. 85A is a graph showing the change in the data amount DA stored inthe read buffer 3721 during operation in 2D playback mode. FIG. 85B is aschematic diagram showing the correspondence between an extent block8510 for playback and a playback path 8520 in 2D playback mode. As shownin FIG. 85B, in accordance with the playback path 8520, the base-viewdata blocks B[n] (n=0, 1, 2, . . . ) in the extent block 8510 are eachread as one 2D extent EXT2D[n] from the BD-ROM disc 101 into the readbuffer 3721. As shown in FIG. 85A, during the read period PR_(2D)[n] foreach 2D extent EXT2D[n], the stored data amount DA increases at a rateequal to R_(UD54)−R_(EXT2D)[n], the difference between the read rateR_(UD54) and the mean transfer rate R_(EXT2D)[n]. A jump J_(2D)[n],however, occurs between two contiguous 2D extents EXT2D[n−1] andEXT2D[n]. Since the reading of two contiguous dependent-view data blocksDn is skipped during the corresponding jump period PJ_(2D)[n], readingof data from the BD-ROM disc 101 is interrupted. Accordingly, the storeddata amount DA decreases at a mean transfer rate R_(EXT2D)[n] duringeach jump period PJ_(2D)[n].

Reading and transfer operations by the BD-ROM drive 3701 are notactually performed continuously, as suggested by the graph in FIG. 85A,but rather intermittently. During the read period PR_(2D)[n] for each 2Dextent, this prevents the stored data amount DA from exceeding thecapacity of the read buffer 3721, i.e. overflow in the read buffer 3721.Accordingly, the graph in FIG. 85A represents what is actually astep-wise increase as an approximated straight increase.

In order to play back 2D video images seamlessly from the extent block8510 shown in FIG. 85B, the following conditions [1] and [2] should bemet.

[1] While data is continuously provided from the read buffer 3721 to thesystem target decoder 4225 during each jump period PJ_(2D)[n], continualoutput from the system target decoder 4225 needs to be ensured. To doso, the following condition should be met: the size S_(EXT2D)[n] of each2D extent EXT2D[n] is the same as the data amount transferred from theread buffer 3721 to the system target decoder 4225 from the read periodPR_(2D)[n] through the next jump period PJ_(2D)[n+1]. If this is thecase, then as shown in FIG. 85A, the stored data amount DA at the end ofthe jump period PJ_(2D)[n+1] does not fall below the value at the startof the read period PR_(2D)[n]. In other words, during each jump periodPJ_(2D)[n], data is continuously provided from the read buffer 3721 tothe system target decoder 4225. In particular, underflow does not occurin the read buffer 3721. In this case, the length of the read periodPR_(2D)[n] equals S_(EXT2D)[n]/R_(UD54), the value obtained by dividingthe size S_(EXT2D)[n] of a 2D extent EXT2D[n] by the read rate R_(UD54).Accordingly, the size S_(EXT2D)[n] of each 2D extent EXT2D[n] should beequal to or greater than the minimum extent size expressed in theright-hand side of expression 1.

$\begin{matrix}{{{S_{{EXT}\; 2D}\lbrack n\rbrack} \geq {\left( {\frac{S_{{EXT}\; 2D}\lbrack n\rbrack}{R_{{UD}\; 54}} + {T_{{JUMP} - {2D}}\lbrack n\rbrack}} \right) \times {R_{{EXT}\; 2D}\lbrack n\rbrack}}}\therefore {{S_{{EXT}\; 2D}\lbrack n\rbrack} \geq {{CEIL}\left( {\frac{R_{{EXT}\; 2D}\lbrack n\rbrack}{8} \times \frac{R_{{UD}\; 54}}{R_{{UD}\; 54} - {R_{{EXT}\; 2D}\lbrack n\rbrack}} \times {T_{{JUMP} - {2\; D}}\lbrack n\rbrack}} \right)}}} & (1)\end{matrix}$

In expression 1, the jump time T_(JUMP-2D)[n] represents the length ofthe jump period PJ_(2D)[n] in seconds. The read rate R_(UD54) and themean transfer rate R_(EXT2D) are both expressed in bits per second.Accordingly, in expression 1, the mean transfer rate R_(EXT2D) isdivided by 8 to convert the size S_(EXT2D)[n] of the 2D extent from bitsto bytes. That is, the size S_(EXT2D)[n] of the 2D extent is expressedin bytes. The function CEIL( ) is an operation to round up fractionalnumbers after the decimal point of the value in parentheses.

[2] Since the capacity of the read buffer 3721 is limited, the maximumvalue of the jump period T_(JUMP-2D)[n] is limited. In other words, evenif the stored data amount DA immediately before a jump period PJ_(2D)[n]is the maximum capacity of the read buffer 3721, if the jump timeT_(JUMP-2D)[n] is too long, the stored data amount DA will reach zeroduring the jump period PJ_(2D)[n], and there is a danger of underflowoccurring in the read buffer 3721. Hereinafter, the time for the storeddata amount DA to decrease from the maximum capacity of the read buffer3721 to zero while data supply from the BD-ROM disc 101 to the readbuffer 3721 has stopped, that is, the maximum value of the jump timeT_(JUMP-2D) that guarantees seamless playback, is referred to as the“maximum jump time T_(JUMP) _(—) _(MAX)”.

In standards of optical discs, the correspondence between jump distancesand maximum jump times is determined from the access speed of theoptical disc drive and other factors. FIG. 86 is an example of acorrespondence table between jump distances S_(JUMP) and maximum jumptimes T_(JUMP) _(—) _(MAX) for a BD-ROM disc. As shown in FIG. 86, jumpdistances S_(JUMP) are represented in units of sectors, and maximum jumptimes T_(JUMP) _(—) _(MAX) are represented in milliseconds. One sectorequals 2048 bytes. When a jump distance S_(JUMP) is zero sectors or iswithin a range of 1-10000 sectors, 10001-20000 sectors, 20001-40000sectors, 40001 sectors- 1/10 of a stroke, and 1/10 of a stroke orgreater, the corresponding maximum jump time T_(JUMP) _(—) _(MAX) is 0msec, 250 msec, 300 msec, 350 msec, 700 msec, and 1400 msec,respectively. When the jump distance S_(JUMP) equals zero sectors, themaximum jump time T_(JUMP) _(—) _(MAX) equals a zero sector transitiontime T_(JUMP0). In the example in FIG. 86, the zero sector transitiontime T_(JUMP0) is considered to be 0 msec.

Based on the above considerations, the jump time T_(JUMP-2D)[n] to besubstituted into expression 1 is the maximum jump time T_(JUMP) _(—)_(MAX) specified for each jump distance by BD-ROM disc standards.Specifically, the jump distance S_(JUMP) between the 2D extentsEXT2D[n−1] and EXT2D[n] is substituted into expression 1 as the jumptime T_(JUMP-2D)[n]. This jump distance S_(JUMP) equals the maximum jumptime T_(JUMP) _(—) _(MAX) that corresponds to the number of sectors fromthe end of the (n+1)^(th) 2D extent EXT2D[n] to the top of the(n+2)^(th) 2D extent EXT2D[n+1] as found in the table in FIG. 86.

Since the jump time T_(JUMP-2D)[n] for the jump between two 2D extentsEXT2D[n] and EXT2D[n+1] is limited to the maximum jump time T_(JUMP)_(—) _(MAX), the jump distance S_(JUMP), i.e. the distance between thetwo 2D extents EXT2D[n] and EXT2D[n+1], is also limited. When the jumptime T_(JUMP) equals a maximum jump time T_(JUMP) _(—) _(MAX), the jumpdistance S_(JUMP) reaches a maximum value, referred to as the “maximumjump distance S_(JUMP) _(—) _(MAX)”. For seamless playback of 2D videoimages, in addition to the size of 2D extents satisfying expression 1,the distance between 2D extents needs to be equal to or less than themaximum jump distance S_(JUMP) _(—) _(MAX).

Within each extent block, the distance between 2D extents equals thesize of a dependent-view data block. Accordingly, this size is limitedto being equal to or less than the maximum jump distance S_(JUMP) _(—)_(MAX). Specifically, when the maximum jump time T_(JUMP) _(—) _(MAX)between 2D extents is limited to the minimum value 250 msec specified inFIG. 86, then the distance between 2D extents, i.e. the size ofdependent-view data blocks, is limited to the corresponding maximum jumpdistance S_(JUMP) _(—) _(MAX)=10000 sectors or less.

When seamlessly playing back two extent blocks arranged on differentrecording layers, a long jump occurs between the (n+1)^(th) 2D extentEXT2D[n] located at the end of the earlier extent block and the(n+2)^(th) 2D extent EXT2D[n+1] located at the top of the later extentblock. This long jump is caused by an operation, such as a focus jump,to switch the recording layer. Accordingly, in addition to the maximumjump time T_(JUMP) _(—) _(MAX)specified in the table in FIG. 86, thetime required for this long jump further includes a “layer switchingtime”, which is the time necessary for an operation to switch therecording layer. This “layer switching time” is, for example, 350 msec.As a result, in expression 1, which the size of the (n+1)^(th) 2D extentEXT2D[n] should satisfy, the jump time T_(JUMP-2D)[n] is determined bythe sum of two parameters TJ[n] and TL[n]: T_(JUMP-2D)[n]=TJ[n]+TL[n].The first parameter TJ[n] represents the maximum jump time T_(JUMP) _(—)_(MAX) specified for the jump distance S_(JUMP) of the long jumpaccording to BD-ROM disc standards. This maximum jump time T_(JUMP) _(—)_(MAX) equals the value, in the table in FIG. 86, corresponding to thenumber of sectors from the end of the (n+1)^(th) 2D extent EXT2D[n] tothe top of the (n+2)^(th) 2D extent EXT2D[n+1]. The second parameterTL[n] represents the layer switching time, for example 350 msec.Accordingly, the distance between two 2D extents EXT2D[n] and EXT2D[n+1]is limited to being equal to or less than the maximum jump distanceS_(JUMP) _(—) _(MAX) corresponding, in the table in FIG. 86, to themaximum jump time T_(JUMP) _(—) _(MAX) of the long jump minus the layerswitching time.

<<Conditions Based on Capability in 3D Playback Mode>>

FIG. 87 is a block diagram showing playback processing in the playbackdevice 102 in 3D playback mode. As shown in FIG. 87, from among theelements shown in FIG. 42, this playback processing system includes theBD-ROM drive 4201, switch 4220, pair of RB1 4221 and RB2 4222, andsystem target decoder 4225. The BD-ROM drive 4201 reads extents SS fromthe BD-ROM disc 101 and transfers the extents SS to the switch 4220 at aread rate R_(UD72). The switch 4220 separates extents SS into base-viewdata blocks and dependent-view data blocks. The base-view data blocksare stored in the RB1 4221, and the dependent-view data blocks arestored in the RB2 4222. The system target decoder 4225 reads sourcepackets from the base-view data blocks stored in the RB1 4221 at abase-view transfer rate R_(EXT1) and reads source packets from thedependent-view data blocks stored in the RB2 4222 at a dependent-viewtransfer rate R_(EXT2). The system target decoder 4225 also decodespairs of read base-view data blocks and dependent-view data blocks intovideo data VD and audio data AD.

The base-view transfer rate R_(EXT1) and the dependent-view transferrate R_(EXT2) equal 192/188 times the mean rate of processing by thesystem target decoder 4225 to extract TS packets respectively from eachsource packet in the base-view data blocks and the dependent-view datablocks. The maximum value R_(MAX1) of the base-view transfer rateR_(EXT1) equals 192/188 times the system rate R_(TS1) for the file 2D.The maximum value R_(MAX2) of the dependent-view transfer rate R_(EXT2)equals 192/188 times the system rate R_(TS2) for the file DEP. Thetransfer rates R_(EXT1) and R_(EXT2) are conventionally represented inbits/second and specifically equal the value of the size of each datablock expressed in bits divided by the extent ATC time. The extent ATCtime equals the time required to transfer all of the source packets inthe data block from the read buffers 4221, 4222 to the system targetdecoder 4225.

The read rate R_(UD72) is conventionally expressed in bits/second and isset at a higher value, e.g. 72 Mbps, than the maximum values R_(MAX1),R_(MAX2) of the transfer rates R_(EXT1), R_(EXT2): R_(UD72)>R_(MAX1),R_(UD72)>R_(MAX2). This prevents underflow in the RB1 4221 and RB2 4222due to decoding processing by the system target decoder 4225 while theBD-ROM drive 4201 is reading an extent SS from the BD-ROM disc 101.

[Seamless Connection Within an Extent Block]

FIGS. 88A and 88B are graphs showing changes in data amounts DA1 and DA2stored in RB1 4221 and RB2 4222 when 3D video images are played backseamlessly from a single extent block. FIG. 88C is a schematic diagramshowing a correspondence between the extent block 8810 and a playbackpath 8820 in 3D playback mode. As shown in FIG. 88C, in accordance withthe playback path 8820, the entire extent block 8810 is read all at onceas one extent SS. Subsequently, the switch 4220 separates the extent SSinto dependent-view data blocks D[k] and base-view data blocks B[k] (k=. . . , n, n+1, n+2, . . . ).

Reading and transfer operations by the BD-ROM drive 4201 are notactually performed continuously, as suggested by the graphs in FIGS. 88Aand 88B, but rather intermittently. During the read periods PR_(D)[k]and PR_(B)[k] for the data blocks D[k], B[k], this prevents overflow inthe RB1 4221 and RB2 4222. Accordingly, the graphs in FIGS. 88A and 88Brepresent what is actually a step-wise increase as an approximatedstraight increase.

As shown in FIGS. 88A and 88B, during the read period PR_(D)[n] of then^(th) dependent-view data block D[n], the stored data amount DA2 in theRB2 4222 increases at a rate equal to R_(UD72)−R_(EXT2)[n], thedifference between the read rate R_(UD72) and a dependent-view transferrate R_(EXT2)[n], whereas the stored data amount DA1 in the RB1 4221decreases at a base-view transfer rate R_(EXT1)[n−1]. As shown in FIG.88C, a zero sector transition J₀ [2n] occurs from the (n+1)^(th)dependent-view data block D[n] to the (n+1)^(th) base-view data blockB[n]. As shown in FIGS. 88A and 88B, during the zero sector transitionperiod PJ₀[n], the stored data amount DA1 in the RB1 4221 continues todecrease at the base-view transfer rate R_(EXT1)[n−1], whereas thestored data amount DA2 in the RB2 4222 decreases at the dependent-viewtransfer rate R_(EXT2)[n].

As further shown in FIGS. 88A and 88B, during the read period PR_(B)[n]of the n^(th) base-view data block B[n], the stored data amount DA1 inthe RB1 4221 increases at a rate equal to R_(UD72)−R_(EXT1)[n], thedifference between the read rate R_(UD72) and a base-view transfer rateR_(EXT1)[n]. On the other hand, the stored data amount DA2 in the RB24222 continues to decrease at the dependent-view transfer rateR_(EXT2)[n]. As further shown in FIG. 88C, a zero sector transitionJ₀[2n+1] occurs from the base-view data block B[n] to the nextdependent-view data block D(n+1). As shown in FIGS. 88A and 88B, duringthe zero sector transition period PJ₀[2n+1], the stored data amount DA1in the RB1 4221 decreases at the base-view transfer rate R_(EXT1)[n],and the stored data amount DA2 in the RB2 4222 continues to decrease atthe dependent-view transfer rate R_(EXT2)[n].

In order to play back 3D video images seamlessly from one extent block8810, the following conditions [3] and [4] should be met.

[3] The size S_(EXT1)[n] of the (n+1)^(th) base-view data block B[n] isat least equal to the data amount transferred from the RB1 4221 to thesystem target decoder 4225 from the corresponding read period PR_(B)[n]until immediately before the read period PR_(B)[n+1] of the nextbase-view data block B[n+1]. In this case, as shown in FIG. 88A,immediately before the read period PR_(B)[n+1] of the next base-viewdata block B[n+1], the stored data amount DA1 in the RB1 4221 does notfall below the amount immediately before the read period PR_(B)[n] ofthe n^(th) base-view data block B[n]. The length of the read periodPR_(B)[n] of the (n+1)^(th) base-view data block B[n] equalsS_(EXT1)[n]/R_(UD72), the value obtained by dividing the sizeS_(EXT1)[n] of this base-view data block B[n] by the read rate R_(UD72).On the other hand, the length of the read period PR_(R)[n+1] of the(n+2)^(th) dependent-view data block D[n+1] equalsS_(EXT2)[n+1]/R_(UD72), the value obtained by dividing the sizeS_(EXT2)[n+1] of this dependent-view data block D[n+1] by the read rateR_(UD72). Accordingly, the size S_(EXT1)[n] of this base-view data blockB[n] should be equal to or greater than the minimum extent sizeexpressed in the right-hand side of expression 2.

$\begin{matrix}{{{S_{{EXT}\; 1}\lbrack n\rbrack} \geq {\left( {\frac{S_{{EXT}\; 1}\lbrack n\rbrack}{R_{{UD}\; 72}} + {T_{{JUMP}\; 0}\left\lbrack {{2n} + 1} \right\rbrack} + \frac{S_{{EXT}\; 2}\left\lbrack {n + 1} \right\rbrack}{R_{{UD}\; 72}} + {T_{{JUMP}\; 0}\left\lbrack {{2n} + 2} \right\rbrack}} \right) \times {R_{{EXT}\; 1}\lbrack n\rbrack}}}\therefore{{S_{{EXT}\; 1}\lbrack n\rbrack}{CEIL}\left\{ {\frac{R_{{EXT}\; 1}\lbrack n\rbrack}{8} \times \frac{R_{{UD}\; 72}}{R_{{UD}\; 72} - {R_{{EXT}\; 1}\lbrack n\rbrack}} \times \left( {{T_{{JUMP}\; 0}\left\lbrack {{2n} + 1} \right\rbrack} + \frac{S_{{EXT}\; 2}\left\lbrack {n + 1} \right\rbrack}{R_{{UD}\; 72}} + {T_{{JUMP}\; 0}\left\lbrack {{2n} + 2} \right\rbrack}} \right)} \right\}}} & (2)\end{matrix}$

[4] The size S_(EXT2)[n] of the (n+1)^(th) dependent-view data blockD[n] is at least equal to the data amount transferred from the RB2 4222to the system target decoder 4225 from the corresponding read periodPR_(R)[n] until immediately before the read period PR_(D)[n+1] of thenext dependent-view data block D[n+1]. In this case, as shown in FIG.88B, immediately before the read period PR_(D)[n+1] of the nextdependent-view data block D[n+1], the stored data amount DA2 in the RB24222 does not fall below the amount immediately before the read periodPR_(D)[n] of the n^(th) dependent-view data block D[n]. The length ofthe read period PR_(D)[n] of the (n+1)^(th) dependent-view data blockD[n] equals S_(EXT2)[n]/R_(UD72), the value obtained by dividing thesize S_(EXT2)[n] of this dependent-view data block D[n] by the read rateR_(UD72). Accordingly, the size S_(EXT2)[n] of this dependent-view datablock D[n] should be equal to or greater than the minimum extent sizeexpressed in the right-hand side of expression 3.

$\begin{matrix}{{{S_{{EXT}\; 2}\lbrack n\rbrack} \geq {\left( {\frac{S_{{EXT}\; 2}\lbrack n\rbrack}{R_{{UD}\; 72}} + {T_{{JUMP}\; 0}\left\lbrack {2n} \right\rbrack} + \frac{S_{{EXT}\; 1}\lbrack n\rbrack}{R_{{UD}\; 72}} + {T_{{JUMP}\; 0}\left\lbrack {{2n} + 1} \right\rbrack}} \right) \times {R_{{EXT}\; 2}\lbrack n\rbrack}}}\therefore{{S_{{EXT}\; 2}\lbrack n\rbrack} \geq {{CEIL}\left\{ {\frac{R_{{EXT}\; 2}\lbrack n\rbrack}{8} \times \frac{R_{{UD}\; 72}}{R_{{UD}\; 72} - {R_{{EXT}\; 2}\lbrack n\rbrack}} \times \left( {{T_{{JUMP}\; 0}\left\lbrack {2n} \right\rbrack} + \frac{S_{EXT1}\left\lbrack {n + 1} \right\rbrack}{R_{{UD}\; 72}} + {T_{{JUMP}\; 0}\left\lbrack {{2n} + 1} \right\rbrack}} \right)} \right\}}}} & (3)\end{matrix}$

[Seamless Connection Between Extent Blocks]

FIG. 89B is a schematic diagram showing an (M+1)^(th) (the letter Mrepresents an integer greater than or equal to 1) extent block 8901 and(M+2)^(th) extent block 8902 and the correspondence between these extentblocks 8901 and 8902 and a playback path 8920 in 3D playback mode. Asshown in FIG. 89B, the two extent blocks 8901 and 8902 are separated bya layer boundary LB or a recording area for other data. In accordancewith the playback path 8920, the entire M^(th) extent block 8901 isfirst read all at once as the (M+1)^(th) extent SS EXTSS[M]. A jump J[M]occurs immediately thereafter. Subsequently, the (M+2)^(th) extent block8902 is read all at once as the (M+1)^(th) extent SS EXTSS[M+1].

FIG. 89A is a graph group showing changes in data amounts DA1 and DA2stored in RB1 4221 and RB2 4222, as well as the changes in the sumDA1+DA2, when 3D video images are continually played back seamlesslyfrom two extent blocks 8901 and 8902. In FIG. 89A, the alternating longand short dashed line indicates changes in the data amount DA1 stored inthe RB1 4221, the dashed line indicates changes in the data amount DA2stored in the RB2 4222, and the solid line indicates changes in the sumDA1+DA2 of the two data amounts. In this graph, the solid line is anapproximation that averages small changes each time a data block isread. Furthermore, the zero sector transition time T_(JUMP0) isconsidered to be “zero seconds”.

As shown in FIG. 89A, during the period PR_(BLK)[M] during which theentire (M+1)^(th) extent block 8901 is read from the BD-ROM disc 101into the RB1 4221 and RB2 4222, the data amounts DA1 and DA2respectively stored in the RB1 4221 and RB2 4222 both increase.Specifically, during the period PR_(BLK)[M] during which the entire(M+1)^(th) extent block 8901 is read, the sum DA1+DA2 of the stored dataamounts increases at a rate equal to the differenceR_(UD72)−R_(EXTSS)[M] between the read rate R_(UD72) and a mean transferrate R_(EXTSS)[M]. This mean transfer rate R_(EXTSS)[M] is assessed asthe value obtained by dividing the size of the entire (M+1)^(th) extentblock 8701, i.e. the size S_(EXTSS)[M] of the (M+1)^(th) extent SSEXTSS[M], by the extent ATC time T_(EXTSS).

At the point the last base-view data block in the (M+1)^(th) extentblock 8901 is read into the RB1 4221, the sum DA1+DA2 of the stored dataamount reaches its maximum value. During the period PJ[M] of theimmediately subsequent jump J[M], the sum DA1+DA2 of the stored dataamount decreases at the mean transfer rate R_(EXTSS)[M]. Accordingly, byadjusting the maximum value of the sum DA1+DA2 of the stored data amountto be sufficiently large, underflow in the RB1 4221 and RB2 4222 duringthe jump J[M] can be prevented. As a result, the two extent blocks 8901and 8902 can be seamlessly connected.

The maximum value of the sum DA1+DA2 of the stored data amount isdetermined by the size of the (M+1)^(th) extent block 8701. Accordingly,in order to seamlessly connect the (M+1)^(th) extent block 8901 to the(M+2)^(th) extent block 8902, the size of the (M+1)^(th) extent block8901, i.e. the size S_(EXTSS)[M] of the (M+1)^(th) extent SS EXTSS[M],should satisfy condition 5.

[5] During the read period PR_(D)[m] of the dependent-view data block Dlocated at the top of the (M+1)^(th) extent block 8901, preloading isperformed (the letter m represents an integer greater than or equal to1). During this preload period PR_(D)[m], the base-view data block Bcorresponding to the dependent-view data block D has not been stored inthe RB1 4221, and thus the dependent-view data block D cannot betransferred from the RB2 4222 to the system target decoder 4225.Accordingly, data in the M^(th) extent block is transferred from the RB24222 to the system target decoder 4225 during the preload periodPR_(D)[m]. This maintains the data provision to the system targetdecoder 4225. Similarly, during the read period PR_(D)[n] of thedependent-view data block D located at the top of the (M+2)^(th) extentblock 8902, preloading is performed (the letter n represents an integergreater than or equal to m+1). Accordingly, during the preload periodPR_(D)[n], continued from the immediately prior jump J[M] period, datain the (M+1)^(th) extent block 8901 is transferred from the RB2 4222 tothe system target decoder 4225. This maintains the data provision to thesystem target decoder 4225. Therefore, in order to prevent underflow inboth RB1 4221 and RB2 4222 during the jump J[M], the extent ATC timeT_(EXTSS) of the (M+1)^(th) extent SS EXTSS[M] should be at least equalto the length of the period from the end time T0 of the preload periodPR_(D)[m] in the (M+1)^(th) extent block 8901 until the end time T1 ofthe preload period PR_(D)[n] in the (M+2)^(th) extent block 8902. Inother words, the size S_(EXTSS)[M] of the (M+1)^(th) extent SS EXTSS[M]should at least be equal to the sum of the data amounts transferred fromthe RB1 4221 and RB2 4222 to the system target decoder 4225 during theperiod T0-T1.

As is clear from FIG. 89A, the length of the period T0-T1 equals the sumof the length of the read period PR_(BLK)[M] of the (M+1)^(th) extentblock 8901, the jump time T_(JUMP)[M] of the jump J[M], and thedifference T_(DIFF)[M] in the lengths of the preload periods PR_(D)[n]and PR_(D)[m] in the extent blocks 8901 and 8002. Furthermore, thelength of the read period PR_(BLK)[M] of the (M+1)^(th) extent block8901 equals S_(EXTSS)[M]/R_(UD72), the value obtained by dividing thesize S_(EXTSS)[M] of the (M+1)^(th) extent SS EXTSS[M] by the read rateR_(UD72). Accordingly, the size S_(EXTSS)[M] of the (M+1)^(th) extent SSEXTSS[M] should be equal to or greater than the minimum extent sizeexpressed in the right-hand side of expression 4.

$\begin{matrix}{{{S_{EXTSS}\lbrack M\rbrack} \geq {\left( {\frac{S_{EXTSS}\lbrack M\rbrack}{R_{{UD}\; 72}} + {T_{JUMP}\lbrack M\rbrack} + {T_{DIFF}\lbrack M\rbrack}} \right) \times {R_{EXTSS}\lbrack M\rbrack}}}\therefore{{S_{EXTSS}\lbrack M\rbrack} \geq {\frac{R_{{UD}\; 72} \times {R_{EXTSS}\lbrack M\rbrack}}{R_{{UD}\; 72} - {R_{EXTSS}\lbrack M\rbrack}} \times \left( {{T_{JUMP}\lbrack M\rbrack} + {T_{DIFF}\lbrack M\rbrack}} \right)}}} & (4)\end{matrix}$

The lengths of the preload periods PR_(D)[m] and PR_(D)[n] respectivelyequal S_(EXT2)[m]/R_(UD72) and S_(EXT2)[n]/R_(UD72), the values obtainedby dividing the sizes S_(EXT2)[m] and S_(EXT2)[n] of the dependent-viewdata block D located at the top of the extent blocks 8901 and 8902 bythe read rate R_(UD72). Accordingly, the difference T_(DIFF) in thelengths of the preload periods PR_(D)[m] and PR_(D)[n] equals thedifference in these values:T_(DIFF)=S_(EXT2)[n]/R_(UD72)−S_(EXT2)[M]/R_(UD72). Note that, like theright-hand side of expressions 1-3, the right-hand side of expression 4may be expressed as an integer value in units of bytes.

Also, when decoding of multiplexed stream data is improved upon asfollows, the difference T_(DIFF) in the right-hand side of expression 4may be considered to be zero. First, the maximum value of the differenceT_(DIFF) throughout the multiplexed stream data, i.e. the worst value ofT_(DIFF), is sought. Next, when the multiplexed stream data is playedback, the start of decoding is delayed after the start of reading by atime equal to the worst value of T_(DIFF).

<<Conditions for Reducing the Capacities of the Read Buffers>>

FIGS. 90A and 90B are graphs showing changes in data amounts DA1 and DA2stored in RB1 4221 and RB2 4222 when 3D video images are played backseamlessly from the two consecutive extent blocks 8901 and 8902 shown inFIG. 89B. As shown in FIG. 90A, the stored data amount DA1 in the RB14221 reaches a maximum value DM1 at the point when the base-view datablock B[n−1] at the end of the (M+1)^(th) extent block 8901 is read intothe RB1 4221. Furthermore, the stored data amount DA1 decreases at thebase-view transfer rate R_(EXT1)[n−1] from the period PJ[M] of theimmediately subsequent jump J[M] through the preload period PR_(D)[n] inthe (M+2)^(th) extent block 8902. Accordingly, to prevent the storeddata amount DA1 from reaching zero before completion of the preloadperiod PR_(D)[n], the maximum value DM1 of the stored data amount DA1should be equal to or greater than the data amount transferred from theRB1 4221 to the system target decoder 4225 during the jump period PJ[M]and the preload period PR_(D)[n]. In other words, the maximum value DM1of the stored data amount DA1 should be greater than or equal to the sumof the length T_(JUMP)[M] of the jump period PJ[M] and the length of thepreload period PR_(D)[n], S_(EXT2)[n]/R_(UD72), multiplied by thebase-view transfer rate R_(EXT1)[n−1]:DM1≧(T_(JUMP)[M]+S_(EXT2)[n]/R_(UD72))×R_(EXT1)[n−1]. When the lengthT_(JUMP)[M] of the jump period PJ[M] equals the maximum jump timeT_(JUMP) _(—) _(MAX) of the jump J[M], and the base-view transfer rateR_(EXT1)[n−1] equals its maximum value R_(MAX1), the maximum value DM1of the stored data amount DA1 is at its largest value. Accordingly, theRB1 4221 is required to have a capacity RB1 equal to or greater than themaximum value DM1 in this case: RB1≧(T_(JUMP) _(—)_(MAX)+S_(EXT2)[n]/R_(UD72))×R_(MAX1).

On the other hand, as shown in FIG. 90B, at the point when reading ofthe end base-view data block B[n−1] in the (M+1)^(th) extent block 8901starts, the stored data amount DA2 in the RB2 4222 reaches its maximumvalue DM2. Furthermore, the stored data amount DA2 decreases at adependent-view transfer rate R_(EXT2)[n−1] from the read period of thebase-view data block B[n−1] through the preload period PR_(D)[n] in the(M+2)^(th) extent block 8902. Accordingly, in order to maintainprovision of data to the system target decoder 4225 through the end ofthe preload period PR_(D)[n], the maximum value DM2 of the stored dataamount DA2 should be equal to or greater than the data amounttransferred from the RB2 4222 to the system target decoder 4225 duringthe read period of the base-view data block B[n−1], the jump periodPJ[M], and the preload period PR_(D)[n]. In other words, the maximumvalue DM2 of the stored data amount DA2 should be greater than or equalto the sum of the length of the read period of the base-view data blockB[n−1] S_(EXT1)[n−1]/R_(UD72), the length T_(JUMP)[M] of the jump periodPJ[M], and the length of the preload period PR_(D)[n],S_(EXT2)[n]/R_(UD72), multiplied by the dependent-view transfer rateR_(EXT2)[n−1]:DM2≧(S_(EXT2)[n−1]/R_(UD72)+T_(JUMP)[M]+S_(EXT2)[n]/R_(UD72))×R_(EXT1)[n−1].When the length T_(JUMP)[M] of the jump period PJ[M] equals the maximumjump time T_(JUMP) _(—) _(MAX) of the jump J[M], and the dependent-viewtransfer rate R_(EXT2)[n−1] equals its maximum value R_(MAX2), themaximum value DM2 of the stored data amount DA2 is at its largest value.Accordingly, the RB2 4222 is required to have a capacity RB2 equal to orgreater than the maximum value DM2 in this case:RB2≧(S_(EXT1)[n−1]/R_(UD72)+T_(JUMP) _(—)_(MAX)+S_(EXT2)[n]/R_(UD72))×R_(MAX2). Furthermore, since anydependent-view data block may be the first data block read duringinterrupt playback, the capacity RB2 of the RB2 4222 should not be lessthan the size of any of the dependent-view data blocks: RB2≧S_(EXT2)[k](the letter k represents an arbitrary integer).

As per the above description, the lower limits of the capacities RB1 andRB2 of the RB1 4221 and RB2 4222 are determined by the sizes S_(EXT1)[k]and S_(EXT2)[k] of the data blocks. Accordingly, in order to economizethe capacities RB1 and RB2, the upper limit of the sizes S_(EXT1)[k] andS_(EXT2)[k] of the data blocks, i.e. the maximum extent size, is limitedvia the following condition [6].

[6] As shown in FIG. 19, the base-view data blocks B[k] in each extentblock 1901-1903 are shared by a file 2D and a file SS. Accordingly, thesize S_(EXT1)[k] of the base-view data blocks B[k] should satisfyexpression 1. On the other hand, in order to reduce the capacity RB1 ofthe RB1 4221 as much as possible, the size S_(EXT1)[k] of the base-viewdata blocks B[k] should be equal to or less than the lower limit of theminimum extent size of 2D extents. In other words, the size S_(EXT1)[k]should be equal to or less than the maximum extent size expressed in theright-hand side of expression 5.

$\begin{matrix}{{S_{{EXT}\; 1}\lbrack k\rbrack} \leq {{CEIL}\left( {\frac{R_{{EXT}\; 1}\lbrack k\rbrack}{8} \times \frac{R_{{UD}\; 54}}{R_{{UD}\; 54} - R_{{MAX}\; 1}} \times T_{{JUMP} - {2\; {D\_ MIN}}}} \right)}} & (5)\end{matrix}$

Note that the lower limit of the minimum extent size of 2D extents isassessed by the expression 1 in which the mean transfer rate R_(EXT2D)included in the denominator of the right-hand side of expression 1 hasbeen replaced with the maximum value R_(MAX). In this expression, thejump time T_(JUMP-2D) _(—) _(MIN) is the minimum value of the jump timenecessary in each extent block 1901-1903, i.e. the minimum value of themaximum jump time T_(JUMP) _(—) _(MAX) between 2D extents. Specifically,the jump time T_(JUMP-2D) _(—) _(MIN) is set to the minimum value 250msec specified in the table in FIG. 86. Meanwhile, the distance between2D extents equals the size S_(EXT2)[k] of a dependent-view data blockD[k]. Accordingly, when the jump time T_(JUMP-2D) _(—) _(MIN) is set to250 msec, the size S_(EXT2)[k] of the dependent-view data block D[k] islimited to the maximum jump distance S_(JUMP) _(—) _(MAX)=10000 sectorsor less corresponding to the maximum jump time T_(JUMP) _(—) _(MAX)=250msec in the table in FIG. 86. In other words, the maximum extent size ofdependent-view data blocks is 10000 sectors.

<<Conclusion>>

To seamlessly play back both 2D video images and 3D video images from aplurality of extent blocks, all of the above conditions [1]-[6] shouldbe satisfied. In particular, the sizes of the data blocks and extentblocks should satisfy the following conditions 1-5.

Condition 1: The size S_(EXT2D) of a 2D extent should satisfy expression1.

Condition 2: The size S_(EXT1) of a base-view data block should satisfyexpression 2.

Condition 3: The size S_(EXT2) a dependent-view data block shouldsatisfy expression 3.

Condition 4: The size S_(EXTSS) of an extent block should satisfyexpression 4.

Condition 5: The size S_(EXT1) of a base-view data block should satisfyexpression 5.

<<Modifications to Condition 1>>

As is clear from the playback path 2101 in 2D playback mode shown inFIG. 21, jumps occur frequently in 2D playback mode. Accordingly, tofurther ensure seamless playback, it is preferable to further add amargin (allowance) to the minimum extent size of the 2D extentsrepresented by the right-hand side of expression 1. However, theaddition of this margin should not change expression 5. This is becauseit may cause the capacity of read buffer to increase. The following arethree methods for adding such a margin without changing expression 5.

The first method adds a margin to the minimum extent size of a 2D extentby replacing the mean transfer rate R_(EXT2D) included in thedenominator of the right-hand side of expression 1 with the maximumvalue R_(MAX). In other words, condition 1 is changed so that the sizeS_(EXT2D) of a 2D extent satisfies expression 6 instead of expression 1.

$\begin{matrix}{{S_{{EXT}\; 2D}\lbrack n\rbrack} \geq {{CEIL}\left( {\frac{R_{{EXT}\; 2D}\lbrack n\rbrack}{8} \times \frac{R_{{UD}\; 54}}{R_{{UD}\; 54} - R_{MAX}} \times {T_{{JUMP} - {2\; D}}\lbrack n\rbrack}} \right)}} & (6)\end{matrix}$

Note that, in the expression 5, an expression which is obtained byreplacing the mean transfer rate R_(EXT2D) included in the denominatorof the right-hand side of expression 1 with the maximum value R_(MAX2D)is used. Accordingly, even if the expression 1 is changed to theexpression 6, the expression 5 is not changed.

The second method adds a margin to the minimum extent size of a 2Dextent by extending the extent ATC time of the 2D extent by ΔT seconds.In other words, condition 1 is changed so that the size S_(EXT2D) of a2D extent satisfies expression 7A or 7B instead of expression 1.

$\begin{matrix}{{S_{{EXT}\; 2D}\lbrack n\rbrack} \geq {{CEIL}\left\{ {\frac{R_{{EXT}\; 2D}\lbrack n\rbrack}{8} \times \left( {{\frac{R_{{UD}\; 54}}{R_{{UD}\; 54} - {R_{{EXT}\; 2D}\lbrack n\rbrack}} \times {T_{{JUMP} - {2D}}\lbrack n\rbrack}} + {\Delta \; T}} \right)} \right\}}} & \left( {7A} \right) \\{{S_{{EXT}\; 2D}\lbrack n\rbrack} \geq {{CEIL}\left\{ {\frac{R_{{EXT}\; 2D}\lbrack n\rbrack}{8} \times \left( {{\frac{R_{{UD}\; 54}}{R_{{UD}\; 54} - R_{MAX}} \times {T_{{JUMP} - {2D}}\lbrack n\rbrack}} + {\Delta \; T}} \right)} \right\}}} & \left( {7B} \right)\end{matrix}$

The extension time ΔT may be determined by the length of a GOP, or bythe upper limit of the number of extents that can be played back duringa predetermined time. For example, if the length of a GOP is one second,ΔT is set to 1.0 seconds. On the other hand, if the upper limit of thenumber of extents that can be played back during a predetermined time inseconds is n, then ΔT is set to the predetermined time/n.

The third method adds a margin to the minimum extent size of the 2Dextent by replacing the mean transfer rate R_(EXT2D) included throughoutthe right-hand side of expression 1 with the maximum value R_(MAX2D). Inother words, condition 1 is changed so that the size S_(EXT2D) of a 2Dextent satisfies expression 8 instead of expression 1.

$\begin{matrix}{{S_{{EXT}\; 2D}\lbrack n\rbrack} \geq {{CEIL}\left( {\frac{R_{MAX}}{8} \times \frac{R_{{UD}\; 54}}{R_{{UD}\; 54} - R_{MAX}} \times {T_{{JUMP} - {2D}}\lbrack n\rbrack}} \right)}} & (8)\end{matrix}$

In this method, an even larger margin can be added to the minimum extentsize. Conversely, however, even when the bit rate of the 2D extent islow, the size needs to be maintained sufficiently large. Accordingly, itis necessary to compare the size of the margin with the efficiency ofrecording data on the BD-ROM disc.

Note that, when the second method is adopted and if more certainty ofthe seamless playback of 2D video images may be prioritized over thereduction of the capacity of the read buffer, expression 5 may bechanged to expression 9.

$\begin{matrix}{{S_{{EXT}\; 1}\lbrack n\rbrack} \leq {{CEIL}\left\{ {\frac{R_{{EXT}\; 2D}\lbrack n\rbrack}{8} \times \left( {{\frac{R_{{UD}\; 54}}{R_{{UD}\; 54} - R_{{MAX}\; 2D}} \times T_{{JUMP} - {2{D\_ MIN}}}} + {\Delta \; T}} \right)} \right\}}} & (9)\end{matrix}$

<Transfer Speed of Stream Data>

FIG. 91 is a block diagram showing the video stream processing systemprovided in the system target decoder 4225 in the 3D playback mode. Asshown in FIG. 91, the processing system includes the pair of sourcedepacketizers 4511 and 4512, the pair of PID filters 4513 and 4514, andthe primary video decoder 4515 which are shown in FIG. 45.

The first source depacketizer 4511 reads source packets from eachbase-view data block in the RB1 4221 at the base-view transfer rateR_(EXT1). The first source depacketizer 4511 further extracts TS packetsfrom the source packets and transfers the TS packets to the first PIDfilter 4513. The mean transfer speed reaches the system rate R_(TS1) forthe file 2D at the maximum. Thus the maximum value R_(MAX1) of thebase-view transfer rate R_(EXT1) equals 192/188 times the system rateR_(TS1). The first PID filter 4513 transmits TS packets including thebase-view video stream to the TB1 4501 in the primary video decoder4515. The TB1 4501 restores PES packets from the TS packets andtransfers the PES packets to the MB1 4502 at the mean speed R_(X1). TheMB1 4502 extracts VAUs of the base-view video stream from the PESpackets, and transfers the VAUs to the EB1 4503 at the mean speedR_(bx1).

The second source depacketizer 4512 reads source packets from eachdependent-view data block in the RB2 4222 at the dependent-view transferrate R_(EXT2). The second source depacketizer 4512 further extracts TSpackets from the source packets and transfers the TS packets to thesecond PID filter 4514. The mean transfer speed reaches the system rateR_(TS2) for the file DEP at the maximum. Thus the maximum value R_(MAX2)of the dependent-view transfer rate R_(EXT2) equals 192/188 times thesystem rate R_(TS2). The second PID filter 4514 transmits TS packetsincluding the dependent-view video stream to the TB2 4508 in the primaryvideo decoder 4515. The TB2 4508 restores PES packets from the TSpackets and transfers the PES packets to the MB2 4509 at the mean speedR_(X2). The MB2 4509 extracts VAUs of the dependent-view video streamfrom the PES packets, and transfers the VAUs to the EB2 4510 at the meanspeed R_(bx2).

The VAUs stored in each of the EB1 4503 and the EB2 4510 are alternatelytransferred to the DEC 4504 by the buffer switch 4506, and decoded touncompressed picture data by the DEC 4504. Here, as shown in FIG. 7, thedependent-view pictures are compressed with reference to the base-viewpictures. Accordingly, the mean bit rate of the dependent-view picturesis lower than that of the base-view pictures. Thus the system rateR_(TS2) for the file DEP can be set to be lower than the system rateR_(TS1) for the file 2D. For example, when the system rate R_(TS1) forthe file 2D is set to be equal to or lower than 45 Mbps, the system rateR_(TS2) for the file DEP can be set to be equal to or lower than 30Mbps: R_(TS1)≦45 Mbps, R_(TS2)≦30 Mbps.

Here, it is assumed that the sum of the system rates R_(TS1) and R_(TS2)is restricted to equal to or smaller than a predetermined thresholdvalue. The predetermined threshold value is set to be equal to orsmaller than the width of a transfer band assigned to the system targetdecoder 4225, for example, equal to 60 Mbps: R_(TS1)+R_(TS2)≦60 Mbps. Inthat case, when the system rate R_(TS1) for the file 2D is set to 45Mbps, the system rate RTS2 for the file DEP is restricted to equal to orsmaller than 15 Mbps: R_(TS1)=45 Mbps, R_(TS2)≦15 Mbps. As far as thebit rate of each video stream is kept to mean value, the restriction onthe sum of the system rates R_(TS1) and R_(TS2) is advantageous forusing the transfer band efficiently. In the actuality, however, thereare cases where the bit rate of the dependent-view video streamtemporarily exceeds the bit rate of the base-view video stream. Forexample, in 3D video images representing natural scenes, such aninversion of bit rates may occur when the base view (i.e. left view)becomes defocused suddenly and only the dependent view (i.e. right view)is focused. In that case, although the base-view transfer rate R_(EXT1)is far lower than the system rate R_(TS1)=45 Mbps, the dependent-viewtransfer rate R_(EXT2) cannot exceed the system rate R_(TS2)≦15 Mbps(accurately, 192/188≈1.02 times thereof. Hereinafter, the coefficient isregarded as “1” unless otherwise noted.). As understood from this, whenthe sum of the system rates R_(TS1) and R_(TS2) is restricted, thedependent-view transfer rate R_(EXT2) cannot respond to a temporaryincrease in the bit rate of the dependent-view video stream.

Such a response to the temporary increase can be realized byrestricting, instead of the sum of the system rates R_(TS1) and R_(TS2),the sum of the base-view transfer rate R_(EXT1) and the dependent-viewtransfer rate R_(EXT2) in units of extents: R_(EXT1)[n]+R_(EXT2)[n]≦60Mbps. The base-view transfer rate R_(EXT1)[n] is a mean value oftransfer speeds at which source packets including the (n+1)^(th) extentEXT1[n] in the file base are transferred, and the dependent-viewtransfer rate R_(EXT2)[n] is a mean value of transfer speeds at whichsource packets including the (n+1)^(th) extent EXT2[n] in the file baseare transferred. FIGS. 92A and 92B are graphs showing temporal changesin the base-view transfer rate R_(EXT1) and the dependent-view transferrate R_(EXT2) in that case, respectively. As shown in FIG. 92A, thebase-view transfer rate R_(EXT1) drops from the maximum valueR_(MAX1)≈45 Mbps at the first time T0, and remains at a low level=15Mbps during a period T_(STR) from the first time T0 to the second timeT1. As indicated by the solid-line graph GR1 in FIG. 92B, thedependent-view transfer rate R_(EXT2) can change in response to thechange of the base-view transfer rate R_(EXT1) in a manner similar tomutual completion. In particular, in the above-mentioned period T_(STR),peak P1 can reach the maximum value R_(MAX2)≈30 Mbps. In this way, whenthe sum of the base-view transfer rate R_(EXT1) and the dependent-viewtransfer rate R_(EXT2) is restricted in units of extents, thedependent-view transfer rate R_(EXT2) can respond to a temporaryincrease in the bit rate of the dependent-view video stream.

In order to use the transfer band assigned to the system target decoder4225 more efficiently in the transfer of the stream data, it ispreferable to set the system rate R_(TS2) for the file DEP to a furtherhigher value. FIG. 92C is a graph showing the temporal change in the sumof the base-view transfer rate R_(EXT1) and the dependent-view transferrate R_(EXT2) shown in FIGS. 92A and 92B. As indicated by the concave CVin the solid-line graph GR3 shown in FIG. 92C, the sum of the base-viewtransfer rate R_(EXT1) and the dependent-view transfer rate R_(EXT2) isbelow a threshold value 60 Mbps during the period T_(STR) from the firsttime T0 to the second time T1. This is because the dependent-viewtransfer rate R_(EXT2) is restricted to equal to or smaller than thesystem rate R_(TS2) (=30 Mbps) for the file DEP, as indicated by thesolid-line graph GR1 in FIG. 92B. As shown in FIG. 92A, because thebase-view transfer rate R_(EXT1) has dropped to 15 Mbps in the periodT_(STR), the transfer band has at least allowance equal to thedifference between the value and the threshold value 60−15=45 Mbps.Thus, the system rate R_(TS2) for the file DEP is set to be in a rangehigher than 30 Mbps, preferably in the same range as the system rateR_(TS1) for the file 2D, for example, equal to or lower than 45 Mbps:R_(TS1)≦45 Mbps, R_(TS2)≦45 Mbps. The dashed-line graphs GR2 and GR4 inFIGS. 92B and 92C show the dependent-view transfer rate R_(EXT2) and thesum of the base-view transfer rate R_(EXT1) and the dependent-viewtransfer rate R_(EXT2) in that case, respectively. As indicated by thedashed-line graph GR2 in FIG. 92B, the peak P2 of the dependent-viewtransfer rate R_(EXT2) can exceed 30 Mbps. As a result, as indicated bythe dashed-line graph GR4 in FIG. 92C, the sum of the base-view transferrate R_(EXT1) and the dependent-view transfer rate R_(EXT2) ismaintained in the proximity of the threshold value 60 Mbps during theperiod T_(STR). In this way, the use efficiency of the transfer band canfurther be improved.

It should be noted here that when the system rate R_(TS2) for the fileDEP is set to a value which is as high as the system rate R_(TS1) forthe file 2D, the sum of them R_(TS1)+R_(TS2) is generally higher thanthe transfer band width of the system target decoder 4225. On the otherhand, since the base-view transfer rate R_(EXT1)[n] and thedependent-view transfer rate R_(EXT2)[n] are each a mean value, even ifa threshold value of the sum of these mean values is provided, the sumof instantaneous values of transfer speeds can exceed the thresholdvalue without restriction. Here, it is assumed as a specific examplethat each of the system rates R_(TS1) and R_(TS2) is set to 45 Mbps, theextent ATC time of each extent is three seconds, and the sum of transferspeeds is maintained to 30 Mbps in the first part (1.5 seconds) of theextent ATC time. In that case, even if each transfer speed increases upto 45 Mbps in the second part (1.5 seconds) of the extent ATC time, thesum of mean values of the transfer speeds in the whole extent is kept at60 Mbps. Accordingly, even if the sum of the base-view transfer rateR_(EXT1)[n] and the dependent-view transfer rate R_(EXT2)[n] isrestricted to equal to or smaller than 60 Mbps, the increase of the sumof instantaneous values of transfer speeds to 90 Mbps cannot be avoided.In this way, by merely restricting the sum of the base-view transferrate R_(EXT1)[n] and the dependent-view transfer rate R_(EXT2)[n], therisk that the transfer band of the system target decoder 4225 issaturated cannot be removed completely.

In order to further reduce the risk that the transfer band of the systemtarget decoder 4225 is saturated, the restriction on the sum of thetransfer speeds may be modified as follows. FIG. 93 is a schematicdiagram showing the relationships between the TS packets that aretransferred in the system target decoder from the source depacketizer tothe PID filter and the ATC times. As shown in FIG. 93, the rectangles9310 in the upper row represent the transfer periods of TS packets TS1#p (p=0, 1, 2, 3, . . . , k, k+1, k+2) constituting the base-viewextent, and the rectangles 9320 in the lower row represent the transferperiods of TS packets TS2 #q (q=0, 1, 2, 3, . . . , m−1, m, m+1)constituting the dependent-view extent. These rectangles 9310, 9320 arearranged in order of transfer of TS packets along the time axis of theATC. The position of the top of each rectangle 9310 and 9320 representsthe transfer start time of the TS packet. The lengths AT1 and AT2 ofeach rectangle 9310 and 9320 represents the amount of time needed forone TS packet to be transferred from the read buffer to the systemtarget decoder. Each time it starts transferring one TS packet from thesource depacketizer, the 3D playback device sets one window (WIN1, WIN2,and WIN3 in FIG. 93) having a predetermined time length (e.g. onesecond) starting from the transfer start time. The 3D playback devicefurther averages each transfer speed of TS1 and TS2 in each of thewindows WIN1, WIN2, and WIN3, and restricts the sum of the averagevalues to a predetermined threshold value or lower. In the example shownin FIG. 93, first the first window WIN1, whose start point is thetransfer start time A1 of TS1 #0, is set, and with regard to TS1 #0-kand TS2 #0-m which are transferred in the window, the sum of thetransfer speeds is restricted to equal to or smaller than the thresholdvalue. Similarly, with regard to TS1 #0-(k+1) and TS2 #0-m which aretransferred in the second window WIN2 whose start point is the transferstart time A2 of TS2 #0, the sum of the transfer speeds is restricted toequal to or smaller than the threshold value, and with regard to TS1#1-(k+1) and TS2 #0-(m+1) which are transferred in the third window WIN3whose start point is the transfer start time A3 of TS1 #1, the sum ofthe transfer speeds is restricted to equal to or smaller than thethreshold value. In this way, the sum of the average transfer speeds ineach window is restricted to equal to or smaller than a predeterminedthreshold value by shifting from one window to another, each having apredetermined length, by the transfer time per TS packet. With thisstructure, as the window becomes shorter, the risk that the transferband of the system target decoder 4225 is saturated becomes lower.

When the system rate R_(TS2) for the file DEP is set to be as high asthe system rate R_(TS1) for the file base, the dependent-view transferrate R_(EXT2) may increase to a similar level. When the dependent-viewtransfer rate R_(EXT2)[n] for the (n+1)^(th) dependent-view extentincreases in such a way, the base-view transfer rate R_(EXT1)[n] for the(n+1)^(th) base-view extent drops to a value that is far lower than themaximum value R_(MAX1). On the other hand, in expression 5 which definesthe maximum extent size, the mean transfer rate R_(EXT2D) included inthe denominator is assessed as the maximum value R_(MAX2D) thereof.Furthermore, the upper limit of the extent ATC time of the (n+1)^(th)base-view extent is a value that is represented as a ratio of itsmaximum extent size to the base-view transfer rate R_(EXT1)[n].Accordingly, the upper limit value is far longer than the actual extentATC time. Since the (n+1)^(th) base-view extent and the (n+1)^(th)dependent-view extent have the extent ATC time in common, the size ofthe dependent-view extent at the maximum is equal to a product of thedependent-view transfer rate R_(EXT2)[n] and the upper limit value ofthe extent ATC time. Since the size is far larger than a value that isactually needed for the seamless playback, the capacity of RB2 cannot befurther reduced. Accordingly, when the system rate R_(TS2) for the fileDEP is set to be as high as the system rate R_(TS1) for the file base,preferably condition 5 for the maximum extent size, namely expression 5is replaced with expression 10.

$\begin{matrix}{{S_{{EXT}\; 1}\lbrack n\rbrack} \leq {{CEIL}\left( {\frac{R_{{EXT}\; 1}\lbrack n\rbrack}{8} \times \frac{R_{{UD}\; 54}}{R_{{UD}\; 54} - {\min\left( {R_{{MAX}\; 2D},{R_{{MAX}\; 1} + R_{{MAX}\; 2} - {R_{{EXT}\; 2}\lbrack n\rbrack}}} \right.}} \times T_{{JUMP} - {2{D\_ MIN}}}} \right)}} & (10)\end{matrix}$

In the right-hand side of expression 10, the maximum value R_(MAX2D) ofthe mean transfer rate for the 2D extent, or the difference between thesum of the transfer rate maximum values R_(MAX1)+R_(MAX2) and thedependent-view transfer rate R_(EXT2), whichever is lower is adopted asthe transfer speed to be included in the denominator. Here, the sum ofthe transfer rate maximum values R_(MAX1)+R_(MAX2) equals 192/188 timesthe sum of the system rates R_(TS1)+R_(TS2). Accordingly, when thedependent-view transfer rate R_(EXT2) increases to a similar level tothe system rate, the maximum extent size is assessed as theabove-described difference. With this structure, the upper limit of theextent ATC time of the base-view extent is restricted to a value that iscloser to the actual extent ATC time. For this reason, the size of thedependent-view extent is restricted to a value that is actually neededfor the seamless playback. In this way, it is possible to maintain thecapacity of RB2 to be sufficiently small.

<Arrangement of Data Blocks when System Rate for File DEP is High>

FIG. 94A is a table showing, with regard to one extent pair, the maximumextent sizes maxS_(EXT1)[n] and maxS_(EXT2)[n] for each combination ofthe base-view transfer rate R_(EXT1)[n] and the dependent-view transferrate R_(EXT2)[n]. Here, “extent pair” refers to a pair of the (n+1)^(th)base-view extent included in the file base and the (n+1)^(th)dependent-view extent included in the file DEP (n=0, 1, 2, . . . ). Themaximum extent sizes maxS_(EXT1)[n] and maxS_(EXT2)[n] are values thatare calculated by using expression 5. As apparent from the fact thatexpression 5 includes the read rate R_(UD54) of the BD-ROM drive in the2D playback mode, the maximum extent sizes maxS_(EXT1)[n] andmaxS_(EXT2)[n] depend on the performance of the BD-ROM drive.Accordingly, the values shown in FIG. 94A are only one example.

As shown in FIG. 94A, when the base-view transfer rate R_(EXT1)[n] is 45Mbps and the dependent-view transfer rate R_(EXT2)[n] is 15 Mbps, themaximum extent size maxS_(EXT2)[n] for the dependent-view extent is 6MB. Conversely, when the base-view transfer rate R_(EXT1)[n] is 15 Mbpsand the dependent-view transfer rate R_(EXT2)[n] is 45 Mbps, the maximumextent size maxS_(EXT2)[n] for the dependent-view extent is 8 MB. Asexplained with reference to FIG. 90, the larger the size of thedependent-view data block located at the top of each extent block is,the larger the capacity required for the read buffer is. Thus it is notpreferable for the dependent-view transfer rate R_(EXT2)[n] to increaseduring the preload period of the extent block because it causes themaximum extent size maxS_(EXT2)[n] for the dependent-view extent toincrease, preventing a further reduction of the capacity of the readbuffer.

Thus, in the extent pair EXT1[n], EXT2[n] located at the top of theextent block, when the dependent-view transfer rate R_(EXT2)[n] exceedsthe base-view transfer rate R_(EXT1)[n], the base-view data block B[n]is arranged before the dependent-view data block D[n]. That is to say,in the extent pair, a small-sized data block is arranged before alarge-sized data block. This enables the capacity of the read buffer tobe kept small as shown in the following.

FIG. 94B is a schematic diagram showing the case where theabove-described arrangement is adopted for two extent blocks 9401 and9402 which are arranged with a layer boundary LB in between. As shown inFIG. 94B, among the extent pairs between the file base 9411 and the fileDEP 9412, the (n+1)^(th) extent pair EXT1[n], EXT2[n] is arranged beforethe M^(th) extent block 9402. In the extent pair, the dependent-viewtransfer rate R_(EXT2)[n] is higher than the base-view transfer rateR_(EXT1)[n], and thus the dependent-view data block D[n] is larger insize than the base-view data block B[n]. Accordingly, in the extentpair, the base-view data block B[n] is arranged before thedependent-view data block D[n]. On the other hand, in the (n−1)^(th),n^(th), and (n+2)^(th) extent pairs EXT1[k], EXT2[k] (k=n−2, n−1, n+1),the dependent-view transfer rate R_(EXT2)[k] is lower than the base-viewtransfer rate R_(EXT1)[k], and thus the dependent-view data block D[k]is smaller in size than the base-view data block B[k]. Accordingly, inthese extent pairs, the dependent-view data block D[k] is arrangedbefore the base-view data block B[k].

FIGS. 95A and 95B are graphs showing changes in amounts DA1, DA2 of datastored in RB1 and RB2, respectively, when 3D video images are playedback seamlessly from the two extent blocks 9401 and 9402 shown in FIG.94B. The graphs G1P, G2P drawn by the solid line show changes in storeddata amounts DA1, DA2 when the base-view data block B[n] is arrangedbefore the dependent-view data block D[n] in the (n+1)^(th) extent pairEXT1[n], EXT2[n] located at the top of the M^(th) extent block 9402. Thegraphs G1Q, G2Q drawn by the dashed line show changes in stored dataamounts DA1, DA2 when the dependent-view data block D[n] is arrangedbefore the base-view data block B[n] in the extent pair EXT1[n],EXT2[n].

As shown in FIG. 95A, the stored data amount DA1 in RB1 reaches amaximum value DM10, DM11 at the point when the base-view data blockB[n−1] at the end of the M^(th) extent block 9401 is read into RB1.Furthermore, the stored data amount DA1 decreases at the base-viewtransfer rate R_(EXT1)[n−1] from the immediately subsequent jump periodPJ[M] through the preload period PR_(B)[n], PR_(D)[n] in the M^(th)extent block 9402. Here, in the (n+1)^(th) extent pair EXT1[n], EXT2[n],the base-view data block B[n] is smaller in size than the dependent-viewdata block D[n]. Accordingly, the length S_(EXT1)[n]/R_(UD72) of thepreload period PR_(B)[n] when the base-view data block B[n] is arrangedbefore the dependent-view data block D[n] is shorter than the lengthS_(EXT2)[n]/R_(UD72) of the preload period PR_(D)[n] of the reversedarrangement. As a result, the maximum value DM11 of the stored dataamount DA1 when the base-view data block B[n] is arranged before thedependent-view data block D[n] is lower than the maximum value DM10 ofthe reversed arrangement.

As shown in FIG. 95B, the stored data amount DA2 in RB2 reaches amaximum value DM20, DM21 at the point when the base-view data blockB[n−1] at the end of the (M−1)^(th) extent block 9401 is read into RB2.Furthermore, the stored data amount DA2 decreases at a dependent-viewtransfer rate R_(EXT2)[n−1] from the read period of the base-view datablock B[n−1] through the preload period PR_(B)[n], PR_(D)[n] in theM^(th) extent block 9402. Here, the length S_(EXT1)[n]/R_(UD72) of thepreload period PR_(B)[n] when the base-view data block B[n] is arrangedbefore the dependent-view data block D[n] is shorter than the lengthS_(EXT2)[n]/R_(UD72) of the preload period PR_(D)[n] of the reversedarrangement. As a result, the maximum value DM21 of the stored dataamount DA2 when the dependent-view data block D[n] is arranged beforethe base-view data block B[n] is lower than the maximum value DM20 ofthe reversed arrangement.

As described above, in the extent pair arranged at the top of the extentblock, it is possible to keep the capacity of the read buffer small byarranging a small-sized data block before a large-sized data block.

Similarly, in the extent pair arranged at a position where an interruptplayback can be started, a small-sized data block is arranged before alarge-sized data block. This enables the capacity of the read buffer tobe kept small. In that case, the order of data blocks may be reversedeven in the extent pair located in the middle of the extent block, aswell as in the extent pair located at the top thereof. FIG. 96A is aschematic diagram showing the syntax of the extent start point for suchan arrangement. This extent start point (Extent_Start_Point), like theextent start points shown in FIGS. 24A and 24B, is set for each of thefile 2D and the file DEP. As shown in FIG. 96A, in the extent startpoint, an extent start flag (is_located_first_in_extent_pair) isassigned to each pair of an extent ID (extent_id) and an SPN(SPN_extent_start).

FIG. 96B is a schematic diagram showing the relationships between thebase-view extent EXT1[k] (k=0, 1, 2, . . . ) belonging to the file baseand the extent start flag indicated by the extent start point. FIG. 96Cis a schematic diagram showing the relationships between thedependent-view extent EXT2[k] belonging to the file DEP and the extentstart flag. FIG. 96D is a schematic diagram showing the relationshipsbetween the extent SS EXTSS[0] belonging to the file SS and the extentblocks on the BD-ROM disc. As shown in FIGS. 96B and 96C, in the extentpair EXT1[k], EXT2[k] having the same extent ID, the value of the extentstart flag is reversely set. In particular, extents whose extent startflag is set to “1” have a smaller number of source packets than extentswhose extent start flag is set to “0”. As shown in FIG. 96D, an extentwhose extent start flag is set to “1” is arranged before an extent whoseextent start flag is set to “0”. In this way, the extent start flagindicates which of EXT1[n] and EXT2[n] in the extent pair is arrangedbefore the other. Thus it is possible to recognize the arrangement ofdata blocks in the extent pair EXT1[n], EXT2[n] from the value of theextent start flag. Accordingly, the playback control unit 4235 cannotify the switch 4220 of the number of source packets from the start ofeach extent SS to each boundary between data blocks by using the extentstart point even if the order of data blocks differs between extentpairs. As a result, the switch 4220 can separate the base-view extentsand dependent-view extents from the extents SS.

When the order of data blocks in each extent pair is constant, theminimum values of the capacities of RB1 and RB2 are calculated by usingFIG. 90 as follows: RB1≧(T_(JUMP) _(—)_(MAX)+S_(EXT2)[n]/R_(UD72))×R_(MAX1), RB2≧max{(S_(EXT1)[n−1]/R_(UD72)+T_(JUMP) _(—)_(MAX)+S_(EXT2)[n]/R_(UD72))×R_(MAX2), S_(EXT2)[n]}. On the other hand,when the order of data blocks may be reversed in the extent pair locatedin the middle of the extent block, the minimum values of the capacitiesof RB1 and RB2 are changed as follows.

FIG. 97C is a schematic diagram showing an arrangement of a data blockwhich requires the largest capacity of RB1 4221. As shown in FIG. 97C,the (M−1)^(th) extent block 9701 and the M^(th) extent block 9702 arearranged with a layer boundary LB in between (the letter M represents aninteger greater than or equal to 2). The (n+1)^(th) extent pair D[n],B[n] is arranged at the top of the M^(th) extent block 9702. Inparticular, the dependent-view data block D[n] is positioned before thebase-view data block B[n] (the letter n represents an integer greaterthan 0). On the other hand, the n^(th) extent pair D[n−1], B[n−1] isarranged at the end of the (M−1)^(th) extent block 9701. In particular,the base-view data block B[n−1] is positioned before the dependent-viewdata block D[n−1].

FIGS. 97A and 97B are graphs showing changes in amounts DA1, DA2 of datastored in RB1 4221 and RB2 4222, respectively, when 3D video images areplayed back seamlessly from the two extent blocks 9701 and 9702 shown inFIG. 97C. As shown in FIG. 97A, the stored data amount DA1 in RB1 4221reaches a maximum value DM1 at the point when the n^(th) base-view datablock B [n−1] is read into RB1 4221. No data block is read into RB1 4221from the immediately subsequent period ΔT1 of reading the dependent-viewdata block D[n−1] through the period ΔT2 in which a long jump across thelayer boundary LB occurs and the preload period ΔT3 in the M^(th) extentblock 9702. Thus during these periods, the stored data amount DA1decreases. In these periods ΔT1-ΔT3, base-view data blocks B[k] (k= . .. , n−3, n−2) up to the (n−1)^(th) base-view data block are transferredat a mean transfer rate R_(EXT1)[ . . . , n−3, n−2], and then the n^(th)base-view data block is transferred at a mean transfer rateR_(EXT1)[n−1]. In order to prevent the stored data amount DA1 fromreaching “0” before the end time point of the preload period ΔT3, thestored data amount DA1 should at least be equal to the sizeS_(EXT1)[n−1] of the base-view data block B[n−1] at the time point whichis before the end time point by the extent ATC time T_(EXT1)[n−1] of then^(th) base-view data block B[n−1]. Accordingly, the maximum value DM1of the stored data amount DA1 should be greater than the sizeS_(EXT1)[n−1] by the data amount R_(EXT1)[ . . . , n−3,n−2]×(ΔT1+ΔT2+ΔT3−T_(EXT1)[n−1]) or more of the data that is transferredfrom RB1 4221 to the system target decoder 4225 in the remaining periodsΔT1+ΔT2+ΔT3−T_(EXT1)[n−1]. That is to say, RB1 4221 is required to havecapacity RB1 that is larger than the maximum value DM1 thereof:RB1≧S_(EXT1)[n−1]+R_(EXT1)[ . . . , n−3,n−2]×(ΔT1+ΔT2+ΔT3−T_(EXT1)[n−1]). Here, the time ΔT2 of a long jump isassessed as the maximum jump time T_(JUMP) _(—) _(MAX) of the long jump.

FIG. 97F is a schematic diagram showing an arrangement of a data blockwhich requires the largest capacity of RB2 4222. As shown in FIG. 97F,the (N−1)^(th) extent block 9703 and the N^(th) extent block 9704 arearranged with a layer boundary LB in between (the letter N represents aninteger greater than or equal to 2). The (n+1)^(th) extent pair D[n],B[n] is arranged at the top of the N^(th) extent block 9704. Inparticular, the dependent-view data block D[n] is positioned after thebase-view data block B[n]. On the other hand, the n^(th) extent pairD[n−1], B[n−1] is arranged at the end of the (N−1)^(th) extent block9703. In particular, the base-view data block B[n−1] is positioned afterthe dependent-view data block D[n−1].

FIGS. 97D and 97E are graphs showing changes in amounts DA1, DA2 of datastored in RB1 4221 and RB2 4222, respectively, when 3D video images areplayed back seamlessly from the two extent blocks 9703 and 9704 shown inFIG. 97F. As shown in FIG. 97E, the stored data amount DA2 in RB2 4222reaches a maximum value DM2 at the point when the n^(th) dependent-viewdata block D[n−1] is read into RB2 4222. No data block is read into RB24222 from the immediately subsequent period ΔT4 of reading the base-viewdata block B[n−1] through the period ΔT5 in which a long jump across thelayer boundary LB occurs and the preload period ΔT6 in the N^(th) extentblock 9704. Thus during these periods, the stored data amount DA2decreases. In these periods ΔT4-ΔT6, dependent-view data blocks D[k] (k=. . . , n−3, n−2) up to the (n−1)^(a)′ dependent-view data block aretransferred at a mean transfer rate R_(EXT2)[ . . . . , n−3, n−2], andthen the n^(th) dependent-view data block is transferred at a meantransfer rate R_(EXT2)[n−1]. In order to prevent the stored data amountDA2 from reaching “0” before the end time point of the preload periodΔT6, the stored data amount DA2 should at least be equal to the sizeS_(EXT2)[n−1] of the dependent-view data block D[n−1] at the time pointwhich is before the end time point by the extent ATC time T_(EXT2)[n−1]of the n^(th) dependent-view data block D[n−1]. Accordingly, the maximumvalue DM2 of the stored data amount DA2 should be greater than the sizeS_(EXT2)[n−1] by the data amount R_(EXT2)[ . . . , n−3,n−2]×(ΔT4+ΔT5+ΔT6−T_(EXT2)[n−1]) or more of the data that is transferredfrom RB2 4222 to the system target decoder 4225 in the remaining periodsΔT4+ΔT5+ΔT6−T_(EXT2)[n−1]. That is to say, RB2 4222 is required to havecapacity RB2 that is larger than the maximum value DM2 thereof:RB2≧S_(EXT2)[n−1]+R_(EXT2)[ . . . , n−3,n−2]×(ΔT4+ΔT5+ΔT6−T_(EXT2)[n−1]). Here, the time ΔT5 of a long jump isassessed as the maximum jump time T_(JUMP) _(—) _(MAX) of the long jump.

When the order of data blocks may be reversed in the extent pair locatedin the middle of the extent block, the conditions 2, 3 for the extentpair, namely expressions 2, 3 are further modified as follows.

FIG. 98C is a schematic diagram showing an extent block 9810 whichincludes in the middle thereof an extent pair in which the order of datablocks is reversed. As shown in FIG. 98C, in the (n+2)^(th) extent pairD[n+1], B[n+1], the dependent-view data block D[n+1] is after thebase-view data block B[n]. In the extent pairs D[n], B[n] and D[n+1],B[n+1] which are before and after thereof, the base-view data blockB[n], B[n+1] is arranged after the dependent-view data block D[n],D[n+1], respectively.

FIGS. 98A and 98B are graphs showing changes in amounts DA1, DA2 of datastored in RB1 4221 and RB2 4222, respectively, when 3D video images areplayed back seamlessly from the extent block 9801 shown in FIG. 98C.Here, the zero sector transition time period is neglected because it issufficiently shorter than the other periods. As shown in FIGS. 98A and98B, during the read period PR_(D)[n] of the (n+1)^(th) dependent-viewdata block D[n], the stored data amount DA2 in the RB2 4222 increases ata rate equal to R_(UD72)−R_(EXT2)[n], the difference between the readrate R_(UD72) and a dependent-view transfer rate R_(EXT2)[n], whereasthe stored data amount DA1 in the RB1 4221 decreases at a base-viewtransfer rate R_(EXT1)[n−1]. During the read period PR_(B)[n] of the(n+1)^(th) base-view data block B[n], the stored data amount DA1 in theRB1 4221 increases at a rate equal to R_(UD72)−R_(EXT1)[n], thedifference between the read rate R_(UD72) and a base-view transfer rateR_(EXT1)[n], whereas the stored data amount DA2 in the RB2 4222decreases at a dependent-view transfer rate R_(EXT2)[n]. During the readperiod PR_(B)[n+1] of the (n+2)^(th) base-view data block B[n+1], thestored data amount DA1 in the RB1 4221 increases at a rate equal toR_(UD72)−R_(EXT1)[n+1], the difference between the read rate R_(UD72)and a base-view transfer rate R_(EXT1)[n+1], whereas the stored dataamount DA2 in the RB2 4222 further decreases at a dependent-viewtransfer rate R_(EXT2)[n+1]. Furthermore, during the read periodPR_(D)[n+1] of the (n+2)^(th) dependent-view data block D[n+1], thestored data amount DA2 in the RB2 4222 increases at a rate equal toR_(UD72)−R_(EXT2)[n+1], the difference between the read rate R_(UD72)and a dependent-view transfer rate R_(EXT2)[n+1], whereas the storeddata amount DA1 in the RB1 4221 decreases at a base-view transfer rateR_(EXT1)[n]. Following this, during the read period PR_(D)[n+2] of the(n+3)^(th) dependent-view data block D[n+2], the stored data amount DA2in the RB2 4222 increases at a rate equal to R_(UD72)−R_(EXT2)[11+2],the difference between the read rate R_(UD72) and a dependent-viewtransfer rate R_(EXT2)[n+2], whereas the stored data amount DA1 in theRB1 4221 decreases at a base-view transfer rate R_(EXT1)[n+1].

In this case, in order to play back 3D video images seamlessly from theextent block 9810, first, the extent ATC time of the (n+1)^(th)dependent-view data block D[n] should be equal to or greater than thetime period from the start time point of the read period PR_(D)[n]thereof to the start time point of the read period PR_(D)[n+1] of thenext dependent-view data block D[n+1]. Next, the extent ATC time of the(n+1)^(th), (n+2)^(th) base-view data block B[n], B[n+1] should be equalto or greater than the time period from the start time point of the readperiod PR_(B)[n], PR_(B)[n+1] thereof to the start time point of theread period PR_(B)[n+2] of the next dependent-view data block B[n+2].These conditions are represented by expressions 2A and 3A instead ofexpressions 2 and 3 when extent B (EXTB) is assumed to be positionedbefore extent A (EXTA) in the n^(th) extent pair.

$\begin{matrix}{{S_{{EXT}\; A}\lbrack n\rbrack} \geq {{CEIL}\left\{ {\frac{R_{{EXT}\; A}\lbrack n\rbrack}{8} \times \frac{R_{{UD}\; 72}}{R_{{UD}\; 72} - {R_{EXTA}\lbrack n\rbrack}} \times \frac{S_{{EXT}\; B}\left\lbrack {n + 1} \right\rbrack}{R_{{UD}\; 72}}} \right\}}} & \left( {2A} \right) \\{{S_{{EXT}\; B}\lbrack n\rbrack} \geq {{CEIL}\left\{ {\frac{R_{{EXT}\; B}\lbrack n\rbrack}{8} \times \frac{R_{{UD}\; 72}}{R_{{UD}\; 72} - {R_{{EXT}\; B}\lbrack n\rbrack}} \times \frac{S_{{EXT}\; A}\left\lbrack {n + 1} \right\rbrack}{R_{{UD}\; 72}}} \right\}}} & \left( {3A} \right)\end{matrix}$

Here, expression 2A is obtained by replacing the size S_(EXT1)[n] of thebase-view data block B[n], the size S_(EXT2)[n+1] of the dependent-viewdata block D[n+1], and the base-view transfer rate R_(EXT1)[n] includedin expression 2 with the size S_(EXTA)[n] of extent A, the sizeS_(EXTB)[n+1] of extent B, and the mean transfer rate R_(EXTA)[n] forextent A, respectively. Expression 3A is obtained by replacing the sizeS_(EXT1)[n] of the base-view data block B[n], the size S_(EXT2)[n] ofthe dependent-view data block D[n], and the dependent-view transfer rateR_(EXT2)[n] included in expression 3 with the size S_(EXTA)[n] of extentA, the size S_(EXTB)[n] of extent B, and the mean transfer rateR_(EXTB)[n] for extent B, respectively. Note that in each of expressions2A and 3A, the zero sector transition time T_(JUMP0) is regarded as “0”.

FIG. 99 is a schematic diagram showing the relationships between anextent block 9900 which includes in the middle thereof an extent pair inwhich the order of data blocks is reversed and AV stream files9910-9920. As shown in FIG. 99, in the third extent pair D[2], B[2], thedependent-view data block D[2] is positioned after the base-view datablock B[2]. In the other extent pairs D[k], B[k] (k=0, 1, 3), thebase-view data block B[k] is positioned after the dependent-view datablock D[k]. Each of the base-view data blocks B[n] (n=0, 1, 2, 3, . . .) belongs to the file base 9911 as one base-view extent EXT1[n]. Each ofthe dependent-view data blocks D[n] belongs to the file DEP 9911 as onedependent-view extent EXT2[n]. The whole extent block 9900 belongs tothe file SS 9920 as one extent SS EXTSS[0]. The base-view data blocksB[n] (n=0, 1, 2, 3, . . . ) further belongs to the file 2D 9910 as the2D extent EXT2D[n]. Here, two consecutive base-view data blocks B[1] andB[2] are referred to as one 2D extent EXT2D [1]. With this structure,the size S_(EXT2D)[1] of the 2D extent EXT2D [1] satisfies expression 1even if the entire size S_(EXT2)[2]+S_(EXT2)[3] of the twodependent-view data blocks D[2], D[3] arranged immediately after thereofis great.

<Separation of a Playback Path Before and After a Layer Boundary>

In FIG. 21, the playback path 2101 in 2D playback mode and the playbackpath 2102 in 3D playback mode both traverse the same base-view datablock B[3] immediately before a long jump J_(LY) that skips over a layerboundary LB. In other words, this base-view data block B[3] is read asthe second 2D extent EXT2D[1] by the playback device 102 in 2D playbackmode and as the last data block in the extent SS EXTSS[1] by theplayback device 102 in 3D playback mode. The data amount to be processedby the system target decoder during the long jump J_(LY) is guaranteedby the size of the base-view data block B[3] via condition 1 in 2Dplayback mode. On the other hand, in 3D playback mode, the data amountis guaranteed by the size of the entire second extent block 1902 viacondition 4. Accordingly, the minimum extent size of the base-view datablock B[3] as required by condition 1 is generally larger than theminimum extent size as per condition 2. Therefore, the capacity of theRB1 4221 has to be larger than the minimum value necessary for seamlessplayback in 3D playback mode. Furthermore, the extent ATC times are thesame for the base-view data block B[3] and the immediately priordependent-view data block D[3]. Accordingly, the size of thedependent-view data block D[3] is generally larger than the minimumextent size required for the data block D[3] as per condition 2.Therefore, the capacity of the RB2 4222 is generally larger than theminimum value necessary for seamless playback in 3D playback mode. Inthe arrangement shown in FIG. 21, two extent blocks 1902 and 1903 canthus be seamlessly connected, but the capacities of the RB1 4221 and RB24222 need to be maintained sufficiently large.

To reduce the capacity of the RB1 4221 and RB2 4222 while stillpermitting seamless playback of video images during a long jump J_(LY),changes may be made in the interleaved arrangement of data blocks beforeand after a position where a long jump J_(LY) is necessary, such as alayer boundary LB, in order to create separate playback paths in 2Dplayback mode and 3D playback mode. These changes are represented, forexample, by the following two types of arrangements 1 and 2. With eitherof the arrangements 1 and 2, the playback path immediately before a longjump J_(LY) traverses different base-view data blocks in each operationmode. As described below, this enables the playback device 102 to easilyperform seamless playback of video images during a long jump J_(LY)while keeping the necessary capacity of the RB1 4221 and RB2 4222 to aminimum.

<<Arrangement 1>>

FIG. 100 is a schematic diagram showing arrangement 1 of a data blockgroup recorded before and after a layer boundary LB on a BD-ROM disc101. As shown in FIG. 100, a first extent block A001 is recorded beforethe layer boundary LB, and a second extent block A002 is recorded afterthe layer boundary LB. In the extent blocks A001 and A002,dependent-view data blocks D[n] and base-view data blocks B[n] form aninterleaved arrangement (n= . . . , 0, 1, 2, 3, . . . ). In particular,the extent ATC times are the same between the n^(th) pair of extentsD[n] and B[n]. In arrangement 1, one base-view data block B[2]_(2D) isfurther placed between the end B[1] of the first extent block A001 andthe layer boundary LB. This base-view data block B[2]_(2D) matchesbit-for-bit with a base-view data block B[2]_(SS) at the top of thesecond extent block A002. Hereinafter, B[2]_(2D) is referred to as a“block exclusively for 2D playback”, and B[2]_(SS) is referred to as a“block exclusively for SS playback”.

The base-view data blocks shown in FIG. 100 can be accessed as extentsin a file 2D A010, i.e. as 2D extents EXT2D[•], with the exception ofthe block exclusively for SS playback B[2]_(SS). For example, thebase-view data block B[0] second from the end of the first extent blockA001, the pair B[1]+B[2]_(2D) of the last base-view data block B[1] andthe block exclusively for 2D playback B[2]_(2D), and the secondbase-view data block B[3] in the second extent block A002 canrespectively be accessed as individual 2D extents EXT2D[0], EXT2D[1],and EXT2D[2]. On the other hand, the dependent-view data blocks D[n] (n=. . . , 0, 1, 2, 3, . . . ) shown in FIG. 100 can each be accessed as asingle extent in the file DEP A012, i.e. as dependent-view extentsEXT2[n].

For the data block groups shown in FIG. 100, cross-linking of AV streamfiles is performed as follows. The entire extent blocks A001 and A002can respectively be accessed as one extent EXTSS[0] and EXTSS[1] in thefile SS A020. Accordingly, the base-view data blocks B[0], B[1], andB[3] in the extent blocks A001 and A002 are shared by the file 2D A010and file SS A020. On the other hand, the block exclusively for 2Dplayback B[2]_(2D) can only be accessed as part of the 2D extentEXT2D[1] located immediately before the layer boundary LB. On the otherhand, the block exclusively for SS playback B[2]_(SS) can only beaccessed as part of the extent SS EXTSS[1] located immediately after thelayer boundary LB. Therefore, the base-view data blocks other than theblock exclusively for 2D playback B[2]_(2D), i.e. B[0], B[1], B[2]_(SS),and B[3], can be extracted from extents SS EXTSS[0], EXTSS[1] as extentsin the file base A011, i.e. base-view extents EXT1[n] (n=0, 1, 2, 3).

FIG. 101 is a schematic diagram showing a playback path A110 in 2Dplayback mode and a playback path A120 in 3D playback mode for the datablock group in arrangement 1 shown in FIG. 100. The playback device 102in 2D playback mode plays back the file 2D A010. Accordingly, as shownby the playback path A110 in 2D playback mode, the base-view data blockB[0] second from the end of the first extent block A001 is read as thefirst 2D extent EXT2D[0], and then reading of the immediately subsequentdependent-view data block D[1] is skipped by a jump J_(2D) 1. Next, apair B[1]+B[2]_(2D) of the last base-view data block B[1] in the firstextent block A001 and the immediately subsequent block exclusively for2D playback B[2]_(2D) is read continuously as the second 2D extentEXT2D[1]. A long jump J_(LY) occurs at the immediately subsequent layerboundary LB, and reading of the three data blocks D[2], B[2]_(SS), andD[3] located at the top of the second extent block A002 is skipped.Subsequently, the second base-view data block B[3] in the second extentblock A002 is read as the third 2D extent EXT2D[2]. On the other hand,the playback device 102 in 3D playback mode plays back the file SS A020.Accordingly, as shown by the playback path A120 in 3D playback mode, theentire first extent block A001 is continuously read as the first extentSS EXTSS[0]. Immediately thereafter, a long jump J_(LY) occurs, andreading of the block exclusively for 2D playback B[2]_(2D) is skipped.Subsequently, the entire second extent block A002 is read continuouslyas the second extent SS EXTSS[1].

As shown in FIG. 100, in 2D playback mode, the block exclusively for 2Dplayback B[2]_(2D) is read, whereas reading of the block exclusively forSS playback B[2]_(SS) is skipped. Conversely, in 3D playback mode,reading of the block exclusively for 2D playback B[2]_(2D) is skipped,whereas the block exclusively for SS playback B[2]_(SS) is read.However, since the data blocks B[2]_(2D) and B[2]_(SS) matchbit-for-bit, the base-view video frames that are played back are thesame in both playback modes. In arrangement 1, the playback path A110 in2D playback mode and the playback path A120 in 3D playback mode aredivided before and after the long jump J_(LY) in this way. Accordingly,unlike the arrangement shown in FIG. 21, the size S_(EXT2D)[1] of the 2Dextent EXT2D[1] located immediately before the layer boundary LB and thesize S_(EXT2)[1] of the immediately preceding dependent-view data blockD[1] can be determined separately as follows.

The size S_(EXT2D)[1] of the 2D extent EXT2D[1] equalsS_(EXT1)[1]+S_(2D), the sum of the size S_(EXT1)[1] of the base-viewdata block B[1] and the size S_(2D) of the block exclusively for 2Dplayback B[2]_(2D). Accordingly, for seamless playback of 2D videoimages, this sum S_(EXT1)[1]+S_(2D) should satisfy expression 1. Themaximum jump time T_(JUMP) _(—) _(MAX) of the long jump J_(LY) issubstituted into the right-hand side of expression 1 as the jump timeT_(JUMP-2D). Next, the number of sectors from the end of the blockexclusively for 2D playback B[2]_(2D) to the first 2D extentEXT2D[2]=B[3] in the second extent block A002 should be equal to or lessthan the maximum jump distance S_(JUMP) _(—) _(MAX) for the long jumpJ_(LY) specified in accordance with the capabilities of the 2D playbackdevice.

On the other hand, for seamless playback of 3D video images, the sizesS_(EXT2)[1] and S_(EXT1)[1] of the dependent-view data block D[1] andbase-view data block B[1] located at the end of the first extent SSEXTSS[0] should satisfy expressions 3 and 2. Regardless of theoccurrence of a long jump J_(LY), a typical value for a zero sectortransition time should be substituted into the right-hand side ofexpressions 3 and 2 as the zero sector transition times T_(JUMP0)[2n+1]and T_(JUMP0)[2n+2]. Next, the size of the first extent SS EXTSS[0]should satisfy condition 4. Furthermore, the number of sectors from theend of this extent SS EXTSS[0] to the top of the extent SS EXTSS[1]should be equal to or less than the maximum jump distance S_(JUMP) _(—)_(MAX) for a long jump J_(LY) specified in accordance with thecapabilities of the 3D playback device.

Within the 2D extent EXT2D[1] located immediately before a layerboundary LB, only the base-view data block B[1] located at the front ofthe 2D extent EXT2D[1] is shared with the first extent SS EXTSS[0].Accordingly, by appropriately enlarging the size S_(2D) of the blockexclusively for 2D playback B[2]_(2D), the size S_(EXT1)[1] of thebase-view data block B[1] can be further limited while keeping the sizeS_(EXT2D)[1]=S_(EXT1)[1]+S_(2D) of the 2D extent EXT2D[1] constant. Inthis case, the extent ATC time of the base-view data block B[1] isshortened. As a result, the size S_(EXT2)[1] of the dependent-view datablock D[1] located immediately before can also be further limited.

Since the block exclusively for SS playback B[2]_(SS) and the blockexclusively for 2D playback B[2]_(2D) match bit for bit, enlarging thesize S_(2D) of the block exclusively for 2D playback B[2]_(2D) enlargesthe size of the dependent-view data block D[2] located immediatelybefore the block exclusively for SS playback B[2]_(SS). However, thissize can be made sufficiently smaller than the size of thedependent-view data block D[3] located immediately before the layerboundary LB shown in FIG. 21. The capacity of the RB1 4221 and RB2 4222can thus be brought even closer to the minimum amount necessary forseamless playback of 3D video images. It is thus possible to set eachdata block in arrangement 1 to be a size at which seamless playback ofboth 2D and 3D video images during a long jump is possible while keepingthe read buffer capacity that is to be guaranteed in the playback device102 to the minimum necessary.

In arrangement 1, duplicate data of the block exclusively for 2Dplayback B[2]_(2D) is arranged in the first extent block A001 as asingle block exclusively for SS playback B[2]_(SS). Alternatively, thisduplicate data may be divided into two or more blocks exclusively for SSplayback.

<<Arrangement 2>>

FIG. 102 is a schematic diagram showing arrangement 2 of a data blockgroup recorded before and after a layer boundary LB on a BD-ROM disc101. As shown by comparing FIG. 102 with FIG. 100, arrangement 2 differsfrom arrangement 1 in that an extent block A202, which includes blocksexclusively for SS playback B[2]_(SS) and B[3]_(SS), is locatedimmediately before a layer boundary LB.

As shown in FIG. 102, a first extent block A201, block exclusively for2D playback (B[2]+B[3])_(2D), and second extent block A202 are locatedbefore a layer boundary LB in this order, and a third extent block A203is located after the layer boundary LB. In the extent blocks A201-A203,dependent-view data blocks D[n] and base-view data blocks B[n] form aninterleaved arrangement (n= . . . , 0, 1, 2, 3, 4, . . . ). Inparticular, the extent ATC times are the same between the n^(th) pair ofextents D[n] and B[n]. In the second extent block A202, stream data iscontinuous with the data blocks D[1] and B[1] located at the end of thefirst extent block A201 and the data blocks D[4] and B[4] located at thetop of the third extent block A203. The base-view data blocks includedin the second extent block A202 are both blocks exclusively for SSplayback, B[2]_(SS) and B[3]_(SS), and the combination of these blocksB[2]_(SS)+B[3]_(SS) matches bit-for-bit with the block exclusively for2D playback (B[2]+B[3])_(2D) located before the second extent blockA202.

Within the base-view data block shown in FIG. 102, data blocks otherthan the blocks exclusively for SS playback B[2]_(SS) and B[3]_(SS) canbe accessed as extents EXT2D[0], EXT2D[1], and EXT2D[2] in a file 2DA210. In particular, the pair of the last base-view data block B[1] andthe block exclusively for 2D playback (B[2]+B[3])_(2D) in the firstextent block A201 can be accessed as a single 2D extent EXT2D[1].Furthermore, the base-view data blocks not located in the second extentblock A202, i.e. the data blocks B[0], B[1], and B[4] in the extentblocks A201 and A203, can also be extracted as extents EXT1[0], EXT1[1],and EXT1[4] in the file base A211 from the extents EXTSS[0] and EXTSS[1]in the file SS A220. Conversely, the block exclusively for 2D playback(B[2]+B[3])_(2D) can only be accessed as part of the 2D extent EXT2D[1].On the other hand, the blocks exclusively for SS playback B[2]_(SS) andB[3]_(SS) can be extracted from the extent SS EXTSS[1] as base-viewextents EXT1[2] and EXT1 [3].

FIG. 103 is a schematic diagram showing a playback path A310 in 2Dplayback mode and a playback path A320 in 3D playback mode for the datablock group in arrangement 2 shown in FIG. 102. The playback device 102in 2D playback mode plays back the file 2D A210. Accordingly, as shownby the playback path A310 in 2D playback mode, the base-view data blockB[0] second from the end of the first extent block A201 is read as thefirst 2D extent EXT2D[0], and then reading of the immediately subsequentdependent-view data block D[1] is skipped by a jump J_(2D) 1. Next, thepair of the last base-view data block B[1] in the first extent blockA201 and the immediately subsequent block exclusively for 2D playback(B[2]+B[3])_(2D) are continuously read as the second 2D extent EXT2D[1].A long jump J_(LY) occurs immediately thereafter, and reading of thesecond extent block A202 and the dependent-view data block D[4] locatedat the top of the third extent block A203 is skipped. Subsequently, thefirst base-view data block B[4] in the third extent block A203 is readas the third 2D extent EXT2D[2]. The playback device 102 in 3D playbackmode plays back the file SS A220. Accordingly, as shown by the playbackpath A320 in 3D playback mode, the entire first extent block A201 iscontinuously read as the first extent SS EXTSS[0]. A jump J_(EX) occursimmediately thereafter, and reading of the block exclusively for 2Dplayback (B[2]+B[3])_(2D) is skipped. Next, the entire second extentblock A202 is read continuously as the second extent SS EXTSS[1].Immediately thereafter, a long jump J_(LY) to skip over a layer boundaryLB occurs. Subsequently, the entire third extent block A203 is readcontinuously as the third extent SS EXTSS[2].

As shown in FIG. 103, in 2D playback mode, the block exclusively for 2Dplayback (B[2]+B[3])_(2D) is read, whereas reading of the blocksexclusively for SS playback B[2]_(SS) and B[3]_(SS) is skipped.Conversely, in 3D playback mode, reading of the block exclusively for 2Dplayback (B[2]+B[3])_(2D) is skipped, whereas the blocks exclusively forSS playback B[2]_(SS) and B[3]_(SS) are read. However, since the blockexclusively for 2D playback (B[2]+B[3])_(2D) matches the entirety of theblocks exclusively for SS playback B[2]_(SS)+B[3]_(SS) bit-for-bit, thebase-view video frames that are played back are the same in bothplayback modes. In arrangement 2, the playback path A310 in 2D playbackmode and the playback path A320 in 3D playback mode are divided beforeand after the long jump J_(LY) in this way. Accordingly, the sizeS_(EXT2D)[1] of the 2D extent EXT2D[1] located immediately before thelayer boundary LB and the size S_(EXT2)[1] of the immediately precedingdependent-view data block D[1] can be determined separately as follows.

The size S_(EXT2D)[1] of the 2D extent EXT_(2D)[1] equalsS_(EXT1)[1]+S_(2D), the sum of the size S_(EXT1)[1] of the base-viewdata block B[1] and the size S_(2D) of the block exclusively for 2Dplayback (B[2]+B[3])_(2D). Accordingly, for seamless playback of 2Dvideo images, this sum S_(EXT1)[1]+S_(2D) should satisfy expression 1.The maximum jump time T_(JUMP) _(—) _(MAX) of the long jump J_(LY) issubstituted into the right-hand side of expression 1 as the jump timeT_(JUMP-2D). Next, the number of sectors from the end of the blockexclusively for 2D playback (B[2]+B[3])_(2D) to the first 2D extentEXT2D[2]=B[4] in the third extent block A203 should be equal to or lessthan the maximum jump distance S_(JUMP) _(—) _(MAX) for the long jumpJ_(LY) specified in accordance with the capabilities of the 2D playbackdevice.

On the other hand, for seamless playback of 3D video images, the sizesS_(EXT2)[1] and S_(EXT1)[1] of the dependent-view data block D[1] andbase-view data block B[1] located at the end of the first extent SSEXTSS[0] should satisfy expressions 3 and 2. Regardless of theoccurrence of a jump J_(EX), a typical value for a zero sectortransition time should be substituted into the right-hand side ofexpressions 3 and 2 as the zero sector transition times T_(JUMP0)[2n+1]and T_(JUMP0)[2n+2]. Next, the sizes S_(EXT2)[3] and S_(EXT1)[3] of thedependent-view data block D[3] and block exclusively for SS playbackB[3]_(SS) located at the end of the second extent SS EXTSS[1] shouldsatisfy expressions 3 and 2. Regardless of the occurrence of a long jumpJ_(LY), a typical value for a zero sector transition time should besubstituted into the right-hand side of expressions 3 and 2 as the zerosector transition times T_(JUMP0)[2n+1] and T_(JUMP0)[2n+2].

Only the base-view data block B[1] located at the front of the 2D extentEXT2D[1] is shared with the extent SS EXTSS[1]. Accordingly, byappropriately enlarging the size S_(2D) of the block exclusively for 2Dplayback (B[2]+B[3])_(2D), the size S_(EXT1)[1] of the base-view datablock B[1] can be further limited while keeping the sizeS_(EXT2D)[1]=S_(EXT1)[1]+S_(2D) of the 2D extent EXT2D[1] constant. As aresult, the size S_(EXT2)[1] of the dependent-view data block D[1]located immediately before can also be further limited.

The blocks exclusively for SS playback B[2]_(SS)+B[3]_(SS) entirelymatch the block exclusively for 2D playback (B[2]+B[3])_(2D) bit forbit. Accordingly, enlarging the size S_(2D) of the block exclusively for2D playback (B[2]+B[3])_(2D) enlarges the sizes of the dependent-viewdata blocks D[2] and D[3] respectively located immediately before theblocks exclusively for SS playback B[2]_(SS) and B[3]_(SS). However,there are two blocks exclusively for SS playback B[2]_(SS) and B[3]_(SS)as compared to one block exclusively for 2D playback (B[2]+B[3])_(2D).As a result, the sizes of each of the blocks exclusively for SS playbackB[2]_(SS) and B[3]_(SS) can be made sufficiently small. The capacity ofthe RB1 4221 and RB2 4222 can thus be further reduced to a minimumamount necessary for seamless playback of 3D video images. It is thuspossible to set each data block in arrangement 2 to be a size at whichseamless playback of both 2D and 3D video images is possible whilekeeping the read buffer capacity that is to be guaranteed in theplayback device 102 to the minimum necessary.

As explained with reference to FIG. 90, the smaller in size the datablock arranged at the end of an extent block positioned immediatelybefore a long jump is, the smaller the lower limit of the capacity ofRB2 4222 is. Accordingly, arrangement 2 is preferably set so as tosatisfy the following two conditions. In that case, in the second extentblock A202, which includes blocks exclusively for 3D playback B[2]_(SS)and B[3]_(SS), each data block is sufficiently small in size. As aresult, the lower limit of the capacity of RB2 4222 can further bereduced.

The first condition is that an upper limit is specified for the size ofthe block exclusively for 2D playback (B[2]+B[3])_(2D) arrangedimmediately before the second extent block A202. For example, as shownin FIG. 103, the size S_(2D) of the block exclusively for 2D playback(B[2]+B[3])_(2D) a is restricted to equal to or smaller than 20000sectors. The upper limit depends on the jump performance of the 2Dplayback device. The second condition is that an upper limit T_(EXT)_(—) _(3D) _(—) _(MAX) is specified for the extent ATC time of theblocks exclusively for 3D playback B[2]_(SS) and B[3]_(SS). That is tosay, the size of the blocks exclusively for 3D playback B[2]_(SS) andB[3]_(SS). satisfies, instead of expression 1, the following expression:S_(EXT1)[n]≦R_(EXT1)[n]×T_(EXT) _(—) _(3D) _(—) _(MAX). The upper limitT_(EXT) _(—) _(3D) _(—) _(MAX) is set to, for example, 0.5 seconds.

FIG. 104 is a schematic diagram showing the relationships between theread time S_(EXT1)[3]/R_(UD72) of the block exclusively for 3D playbackB[3]_(SS) located at the end of the second extent block A202 and thelower limit of the capacity of RB2 4222. As shown in FIG. 104, thestored data amount DA2 in the RB2 4222 reaches a maximum value DM2 atthe point at the start of reading the block exclusively for 3D playbackB[3]_(SS). The maximum value DM2 is equal to or greater than a valuethat is obtained by multiplying the dependent-view transfer rateR_(EXT2)[3] by the sum of the length S_(EXT1)[3]/R_(UD72) of the readperiod of the block exclusively for 3D playback B[3]_(SS), the timerequired for long jump T_(LY), and the length S_(EXT2)[4]/R_(UD72) ofthe preload period:DM2≧(S_(EXT1)[3]/R_(UD72)+T_(LY)+S_(EXT2)[4]/R_(UD72))×R_(EXT2)[3].Accordingly, if the size of the block exclusively for 3D playbackB[3]_(SS) is a value S_(L)[3] which is larger than that, the lengthS_(L)[3]/R_(UD72) of the read period increases. Thus, as shown by thedashed line in FIG. 104, the maximum value DM20 of the stored dataamount DA2 in the RB2 4222 increases. For this reason, theabove-described two conditions are used to restrict the size of each ofthe blocks exclusively for 3D playback B[2]_(SS), B[3]_(SS) to a smallvalue. This enables the lower limit of the capacity of RB2 4222 toreduced further.

Note that, in order for condition 4 to be satisfied, the size of thedata block positioned at the top of each extent block, namely the lengthof the preload period, needs to be maintained sufficiently large. Thuswith regard to the block exclusively for 3D playback positioned at thetop of the extent block, the extent ATC time thereof may exceed theupper limit T_(EXT) _(—) _(3D) _(—) _(MAX).

Arrangement 2 may be provided at a position where an interrupt playbackcan be started, as well as before the layer boundary LB. In FIG. 102,the vertices of the triangles A230, A231, and A232 indicate positionswhere an interrupt playback can be started, namely positions on theBD-ROM disc where entry points are recorded. The entry point indicatedby the white triangle A230 represents a position where an interruptplayback can be started in the 2D playback mode. The entry pointsindicated by the black triangles A231 and A232 represent positions wherean interrupt playback can be started in the 3D playback mode. Each ofthe blocks exclusively for 3D playback B[2]_(SS), B[3]_(SS) is smallersufficiently in size than the block exclusively for 2D playback(B[2]+B[3])_(2D), and thus the size of the corresponding dependent-viewdata block D[2], D[3] is sufficiently small. As a result, when aninterrupt playback is performed in the 3D playback mode, the timerequired from the start of access to the entry point A231, A232 to thestart of decoding of the data block D[2], B[2]_(SS) is short. That is tosay, an interrupt playback in the 3D playback mode is started up fast.

In arrangement 2, duplicate data of the block exclusively for 2Dplayback (B[2]+B[3])_(2D) is divided into two blocks exclusively for SSplayback B[2]_(SS) and B[3]_(SS). Alternatively, the duplicate data maybe one block exclusively for SS playback or may be divided into three ormore blocks exclusively for SS playback.

<Extent Pair Flag>

FIG. 105 is a schematic diagram showing entry points A510 and A520 setfor extents EXT1[k] and EXT2[k] (the letter k represents an integergreater than or equal to j) in a file base A501 and a file DEP A502. Theentry point A510 in the file base A501 is defined by the entry map inthe 2D clip information file, and the entry point A520 in the file DEPA502 is defined by the entry map in the dependent-view clip informationfile. Each entry point A510 and A520 particularly includes an extentpair flag. When an entry point in the file base A501 and an entry pointin the file DEP A502 indicate the same PTS, an “extent pair flag”indicates whether or not the extents in which these entry points are setEXT1[i] and EXT2[j] are in the same order from the top of the files A501and A502 (i=j or i j). As shown in FIG. 105, the PTS of the first entrypoint A530 set in the (n+1)^(th) (the letter n represents an integergreater than or equal to 1) base-view extent EXT1[n] equals the PTS ofthe last entry point A540 set in the (n−1)^(th) dependent-view extentEXT2[n−1]. Accordingly, the value of the extent pair flag for the entrypoints A530 and A540 is set to “0”. Similarly, the PTS of the last entrypoint A531 set in the (n+1)^(th) base-view extent EXT1[n] equals the PTSof the first entry point A541 set in the (n+1)^(th) dependent-viewextent EXT2[n+1]. Accordingly, the value of the extent pair flag for theentry points A531 and A541 is set to “0”. For other entry points A510and A520, when the PTSs are equal, the order of the extents EXT1[•] andEXT2[•] in which these points are set is also equal, and thus the valueof the extent pair flag is set to “1”.

When the playback device 102 in 3D playback mode begins interruptplayback, it refers to the extent pair flag in the entry point of theplayback start position. When the value of the flag is “1”, playbackactually starts from that entry point. When the value is “0”, theplayback device 102 searches, before or after that entry point, foranother entry point that has an extent pair flag with a value of “1”.Playback starts from that other entry point. This ensures that then^(th) dependent-view extent EXT2[n] is read before the n^(th) nbase-view extent EXT1[n]. As a result, interrupt playback can besimplified.

The presentation time corresponding to the distance between entry pointshaving an extent pair flag=0 may be limited to be no greater than aconstant number of seconds. For example, the time may be limited to beless than or equal to twice the maximum value of the presentation timefor one GOP. At the start of interrupt playback, this can shorten thewait time until playback begins, which is caused by searching for anentry point having an extent pair flag=1. Alternatively, the value ofthe extent pair flag for the entry point following an entry point withan extent pair flag=0 may be limited to a value of “1”. An angleswitching flag may also be used as a substitute for an extent pair flag.An “angle switching flag” is a flag prepared within the entry map forcontent that supports multi-angle. The angle switching flag indicatesthe angle switching position within multiplexed stream data (see belowfor a description of multi-angle).

<Matching Playback Periods Between Data Blocks>

For pairs of data blocks with equal extent ATC times, the playbackperiod may also match, and the playback time of the video stream may beequal. In other words, the number of VAUs may be equal between thesedata blocks. The significance of such equality is explained below.

FIG. 106A is a schematic diagram showing a playback path when extent ATCtimes and playback times of the video stream differ between contiguousbase-view data blocks and dependent-view data blocks. As shown in FIG.106A, the playback time of the top base-view data block B[0] is fourseconds, and the playback time of the top dependent-view data block D[0]is one second. In this case, the section of the base-view video streamthat is necessary for decoding of the dependent-view data block D[0] hasthe same playback time as the dependent-view data block D[0].Accordingly, to save read buffer capacity in the playback device, it ispreferable, as shown by the arrow ARW1 in FIG. 106A, to have theplayback device alternately read the base-view data block B[0] and thedependent-view data block D[0] by the same amount of playback time, forexample one second at a time. In that case, however, as shown by thedashed lines in FIG. 106A, jumps occur during read processing. As aresult, it is difficult to cause read processing to keep up withdecoding processing, and thus it is difficult to stably maintainseamless playback.

FIG. 106B is a schematic diagram showing a playback path when theplayback times of the video stream are equal for contiguous base-viewand dependent-view data blocks. As shown in FIG. 106B, the playback timeof the video stream between a pair of adjacent data blocks may be thesame. For example, for the pair of the top data blocks B[0] and D[0],the playback times of the video stream both equal one second, and theplayback times of the video stream for the second pair of data blocksB[1] and D[1] both equal 0.7 seconds. In this case, during 3D playbackmode, the playback device reads data blocks B[0], D[0], B[1], D[1], . .. in order from the top, as shown by arrow ARW2 in FIG. 106B. By simplyreading these data blocks in order, the playback device can smoothlyread the main TS and sub-TS alternately in the same increments ofplayback time. In particular, since no jump occurs during readprocessing, seamless playback of 3D video images can be stablymaintained.

If the extent ATC time is actually the same between contiguous base-viewand dependent-view data blocks, jumps do not occur during reading, andsynchronous decoding can be maintained. Accordingly, even if theplayback period or the playback time of the video stream are not equal,the playback device can reliably maintain seamless playback of 3D videoimages by simply reading data block groups in order from the top, as inthe case shown in FIG. 106B.

The number of any of the headers in a VAU, and the number of PESheaders, may be equal for contiguous base-view and dependent-view datablocks. These headers are used to synchronize decoding between datablocks. Accordingly, if the number of headers is equal between datablocks, it is relatively easy to maintain synchronous decoding, even ifthe number of VAUs is not equal. Furthermore, unlike when the number ofVAUs is equal, all of the data in the VAUs need not be multiplexed inthe same data block. Therefore, there is a high degree of freedom formultiplexing stream data during the authoring process of the BD-ROM disc101.

The number of entry points may be equal for contiguous base-view anddependent-view data blocks. Referring again to FIG. 105, in the filebase A501 and the file DEP A502, the extents EXT1[n] and EXT2[n],located in the same order from the top, have the same number of entrypoints A510 and A520, after excluding the entry points A530, A540, A531,A541 with an extent pair flag=0. Whether jumps are present differsbetween 2D playback mode and 3D playback mode. When the number of entrypoints is equal between data blocks, however, the playback time issubstantially equal. Accordingly, it is easy to maintain synchronousdecoding regardless of jumps. Furthermore, unlike when the number ofVAUs is equal, all of the data in the VAUs need not be multiplexed inthe same data block. Therefore, there is a high degree of freedom formultiplexing stream data during the authoring process of the BD-ROM disc101.

<Multi-Angle>

FIG. 107A is a schematic diagram showing a playback path for multiplexedstream data supporting multi-angle. As shown in FIG. 107A, three typesof pieces of stream data L, R, and D respectively for a base view, rightview, and depth map are multiplexed in the multiplexed stream data. Forexample, in L/R mode the base-view and right-view pieces of stream dataL and R are played back in parallel. Furthermore, pieces of stream dataAk, Bk, and Ck (k=0, 1, 2, . . . , n) for different angles (viewingangles) are multiplexed in the section played back during a multi-angleplayback period P_(ANG). The stream data Ak, Bk, and Ck for differentangles is divided into sections for which the playback time equals theangle change interval. Furthermore, stream data for the base view, rightview and depth map is multiplexed in each of the pieces of data Ak, Bk,and Ck. During the multi-angle playback period P_(ANG), playback can beswitched between the pieces of stream data Ak, Bk, and Ck for differentangles in response to user operation or instruction by an applicationprogram.

FIG. 107B is a schematic diagram showing a data block group A701recorded on a BD-ROM disc and a corresponding playback path A702 in L/Rmode. This data block group A701 includes the pieces of stream data L,R, D, Ak, Bk, and Ck shown in FIG. 107A. As shown in FIG. 107B, in thedata block group A701, in addition to the regular pieces of stream dataL, R, and D, the pieces of stream data Ak, Bk, and Ck for differentangles are recorded in an interleaved arrangement. In L/R mode, as shownin the playback path A702, the right-view and base-view data blocks Rand L are read, and reading of the depth map data blocks D is skipped byjumps. Furthermore, from among the pieces of stream data Ak, Bk, and Ckfor different angles, the data blocks for the selected angles A0, B1, .. . , Cn are read, and reading of other data blocks is skipped by jumps.

FIG. 107C is a schematic diagram showing an extent block formed bystream data Ak, Bk, and Ck for different angles. As shown in FIG. 107C,the pieces of stream data Ak, Bk, and Ck for each angle are composed ofthree types of data blocks L, R, and D recorded in an interleavedarrangement. In L/R mode, as shown by the playback path A702, from amongthe pieces of stream data Ak, Bk, and Ck for different angles,right-view and base-view data blocks R and L are read for selectedangles A0, B1, . . . , Cn. Conversely, reading of the other data blocksis skipped by jumps.

Note that in the pieces of stream data Ak, Bk, and Ck for each angle,the stream data for the base view, right view, and depth map may bestored as a single piece of multiplexed stream data. However, therecording rate has to be limited to the range of the system rate forwhich playback is possible in the 2D playback device. Also, the numberof pieces of stream data (TS) to be transferred to the system targetdecoder differs between such pieces of multiplexed stream data andmultiplexed stream data for other 3D video images. Accordingly, each PIin the 3D playlist file may include a flag indicating the number of TSto be played back. By referring to this flag, the 3D playback device canswitch between these pieces of multiplexed stream data within one 3Dplaylist file. In the PI that specifies two TS for playback in 3Dplayback mode, this flag indicates 2TS. On the other hand, in the PIthat specifies a single TS for playback, such as the above pieces ofmultiplexed stream data, the flag indicates 1TS. The 3D playback devicecan switch the setting of the system target decoder in accordance withthe value of the flag. Furthermore, this flag may be expressed by thevalue of the connection condition (CC). For example, a CC of “7”indicates a transition from 2TS to 1TS, whereas a CC of “8” indicates atransition from 1TS to 2TS.

FIG. 108 is a schematic diagram showing (i) a data block group A801constituting a multi-angle period and (ii) a playback path A810 in 2Dplayback mode and playback path A820 in L/R mode that correspond to thedata block group A801. As shown in FIG. 108, this data block group A801is formed by three types of angle change sections ANG1 #k, ANG2 #k, andANG3 #k (k=1, 2, . . . , 6, 7) in an interleaved arrangement. An “anglechange section” is a group of consecutive data blocks in which streamdata for video images seen from a single angle is stored. The angle ofvideo images differs between different types of angle change sections.The k^(th) sections of each type of angle change section ANG1 #k, ANG2#k, and ANG3 #k are contiguous. Each angle change section ANGm #k (m=1,2, 3) is formed by one extent block, i.e. is referred to as one extentSS EXTSS[k] (k=10, 11, . . . , 23). The capacity of the read buffer canthus be reduced as compared to when a plurality of angle change sectionsform one extent SS EXTSS[k]. Furthermore, each extent block includes onedependent-view data block R and one base-view data block L. This pair ofdata blocks R and L is referred to as a pair of the n^(th)dependent-view extent EXT2[n] and the n^(th) base-view extent EXT1[n](the letter n represents an integer greater than or equal to 0).

The size of each extent block satisfies conditions 1-4. In particular,the jump that should be taken into consideration in condition 1 is thejump J_(ANG-2D) to skip reading of other angle change sections, as shownby the playback path A810 in 2D playback mode. On the other hand, thejump that should be taken into consideration in condition 4 is the jumpJ_(ANG-LR) to skip reading of other angle change sections, as shown bythe playback path A820 in L/R mode. As shown by the playback paths A810and A820, both of these jumps J_(ANG-2D) and J_(ANG-LR) generallyinclude an angle switch, i.e. a switch between the types of angle changesection to be read.

Further referring to FIG. 108, each angle change section includes onebase-view data block L. Accordingly, the extent ATC time of thebase-view extent EXT1[•] is limited to being no greater than the maximumvalue T_(ANG) of the length of the angle change section. For example, inorder to make it possible to switch angles at a rate of once every twoseconds of presentation time, the maximum value T_(ANG) of the length ofthe angle change section has to be limited to two seconds. As a result,the extent ATC time of the base-view extent EXT1[•] is limited to twoseconds or less. Therefore, condition 5 is changed so that the sizeS_(EXT1) of the base-view extent satisfies expression 11 instead ofexpression 5.

$\begin{matrix}{{S_{{EXT}\; 1}\lbrack k\rbrack} \leq {\max \left( {{{R_{{EXT}\; 1}\lbrack k\rbrack} \times \frac{R_{{UD}\; 54}}{R_{{UD}\; 54} - R_{{MAX}\; 1}} \times T_{{JUMP} - {2{D\_ MIN}}}},{{R_{{EXT}\; 1}\lbrack k\rbrack} \times T_{ANG}}} \right)}} & (11)\end{matrix}$

Note that the right-hand side of expression 9 may be compared instead ofthe right-hand side of expression 5. Also, in a similar manner to theextension time ΔT of the extent ATC time of the 2D extent shown inexpression 7A or 7B, the maximum value T_(ANG) of the length of theangle change section may be determined by the length of a GOP, or by theupper limit of the number of extents that can be played back during apredetermined time. Furthermore, the extension time ΔT may be set to “0”in the multi-angle.

<Data Distribution via Broadcasting or Communication Circuit>

The recording medium according to the embodiments of the presentinvention may be, in addition to an optical disc, a general removablemedium available as a package medium, such as a portable semiconductormemory device, including an SD memory card. Also, the above embodimentsdescribe an example of an optical disc in which data has been recordedbeforehand, namely, a conventionally available read-only optical discsuch as a BD-ROM or a DVD-ROM. However, the embodiments of the presentinvention are not limited in this way. For example, when a terminaldevice writes 3D video content that has been distributed viabroadcasting or a network onto a conventionally available writableoptical disc such as a BD-RE or a DVD-RAM, arrangement of the extentsaccording to Embodiment 1 may be used. The terminal device may beincorporated in a playback device or may be a device different from theplayback device.

<Playback of Semiconductor Memory Card>

The following describes a data reading unit of a playback device in thecase where a semiconductor memory card is used as the recording mediumaccording to the embodiments of the present invention instead of anoptical disc.

The part of the playback device that reads data from an optical disc iscomposed of, for example, an optical disc drive. Conversely, the part ofthe playback device that reads data from a semiconductor memory card iscomposed of an exclusive interface (I/F). Specifically, a card slot isprovided with the playback device, and the I/F is mounted in the cardslot. When the semiconductor memory card is inserted into the card slot,the semiconductor memory card is electrically connected with theplayback device via the I/F. Furthermore, the data is read from thesemiconductor memory card to the playback device via the I/F.

<Copyright Protection Technique for Data Stored in BD-ROM Disc>

The mechanism for protecting copyright of data recorded on a BD-ROM discis now described as an assumption for the following supplementaryexplanation.

From a standpoint, for example, of improving copyright protection orconfidentiality of data, there are cases in which a part of the datarecorded on the BD-ROM is encrypted. The encrypted data is, for example,a video stream, an audio stream, or other stream. In such a case, theencrypted data is decoded in the following manner.

The playback device has recorded thereon beforehand a part of datanecessary for generating a “key” to be used for decoding the encrypteddata recorded on the BD-ROM disc, namely, a device key. On the otherhand, the BD-ROM disc has recorded thereon another part of the datanecessary for generating the “key”, namely, a media key block (MKB), andencrypted data of the “key”, namely, an encrypted title key. The devicekey, the MKB, and the encrypted title key are associated with oneanother, and each are further associated with a particular ID writteninto a BCA 201 recorded on the BD-ROM disc 101 shown in FIG. 2, namely,a volume ID. When the combination of the device key, the MKB, theencrypted title key, and the volume ID is not correct, the encrypteddata cannot be decoded. In other words, only when the combination iscorrect, the above-mentioned “key”, namely the title key, can begenerated. Specifically, the encrypted title key is first decryptedusing the device key, the MKB, and the volume ID. Only when the titlekey can be obtained as a result of the decryption, the encrypted datacan be decoded using the title key as the above-mentioned “key”.

When a playback device tries to play back the encrypted data recorded onthe BD-ROM disc, the playback device cannot play back the encrypted dataunless the playback device has stored thereon a device key that has beenassociated beforehand with the encrypted title key, the MKB, the device,and the volume ID recorded on the BD-ROM disc. This is because a keynecessary for decoding the encrypted data, namely a title key, can beobtained only by decrypting the encrypted title key based on the correctcombination of the MKB, the device key, and the volume ID.

In order to protect the copyright of at least one of a video stream andan audio stream that are to be recorded on a BD-ROM disc, a stream to beprotected is encrypted using the title key, and the encrypted stream isrecorded on the BD-ROM disc. Next, a key is generated based on thecombination of the MKB, the device key, and the volume ID, and the titlekey is encrypted using the key so as to be converted to an encryptedtitle key. Furthermore, the MKB, the volume ID, and the encrypted titlekey are recorded on the BD-ROM disc. Only a playback device storingthereon the device key to be used for generating the above-mentioned keycan decode the encrypted video stream and/or the encrypted audio streamrecorded on the BD-ROM disc using a decoder. In this manner, it ispossible to protect the copyright of the data recorded on the BD-ROMdisc.

The above-described mechanism for protecting the copyright of the datarecorded on the BD-ROM disc is applicable to a recording medium otherthan the BD-ROM disc. For example, the mechanism is applicable to areadable and writable semiconductor memory device and in particular to aportable semiconductor memory card such as an SD card.

<Recording Data on a Recording Medium Through Electronic Distribution>

The following describes processing to transmit data, such as an AVstream file for 3D video images (hereinafter, “distribution data”), tothe playback device according to the embodiments of the presentinvention via electronic distribution and to cause the playback deviceto record the distribution data on a semiconductor memory card. Notethat the following operations may be performed by a specialized terminaldevice for performing the processing instead of the above-mentionedplayback device. Also, the following description is based on theassumption that the semiconductor memory card that is a recordingdestination is an SD memory card.

The playback device includes the above-described card slot. An SD memorycard is inserted into the card slot. The playback device in this statefirst transmits a transmission request of distribution data to adistribution server on a network. At this point, the playback devicereads identification information of the SD memory card from the SDmemory card and transmits the read identification information to thedistribution server together with the transmission request. Theidentification information of the SD memory card is, for example, anidentification number specific to the SD memory card and, morespecifically, is a serial number of the SD memory card. Theidentification information is used as the above-described volume ID.

The distribution server has stored thereon pieces of distribution data.Distribution data that needs to be protected by encryption such as avideo stream and/or an audio stream has been encrypted using apredetermined title key. The encrypted distribution data can bedecrypted using the same title key.

The distribution server stores thereon a device key as a private keycommon with the playback device. The distribution server further storesthereon an MKB in common with the SD memory card. Upon receiving thetransmission request of distribution data and the identificationinformation of the SD memory card from the playback device, thedistribution server first generates a key from the device key, the MKB,and the identification information and encrypts the title key using thegenerated key to generate an encrypted title key.

Next, the distribution server generates public key information. Thepublic key information includes, for example, the MKB, the encryptedtitle key, signature information, the identification number of the SDmemory card, and a device list. The signature information includes forexample a hash value of the public key information. The device list is alist of devices that need to be invalidated, that is, devices that havea risk of performing unauthorized playback of encrypted data included inthe distribution data. The device list specifies the device key and theidentification number for the playback device, as well as anidentification number or function (program) for each element in theplayback device such as the decoder.

The distribution server transmits the distribution data and the publickey information to the playback device. The playback device receives thedistribution data and the public key information and records them in theSD memory card via the exclusive I/F of the card slot.

Encrypted distribution data recorded on the SD memory card is decryptedusing the public key information in the following manner, for example.First, three types of checks are performed as authentication of thepublic key information. These checks may be performed in any order.

(1) Does the identification information of the SD memory card includedin the public key information match the identification number stored inthe SD memory card inserted into the card slot?

(2) Does a hash value calculated based on the public key informationmatch the hash value included in the signature information?

(3) Is the playback device excluded from the device list indicated bythe public key information? Specifically, is the device key of theplayback device excluded from the device list?

If at least any one of the results of the checks (1) to (3) is negative,the playback device stops decryption processing of the encrypted data.Conversely, if all of the results of the checks (1) to (3) areaffirmative, the playback device authorizes the public key informationand decrypts the encrypted title key included in the public keyinformation using the device key, the MKB, and the identificationinformation of the SD memory card, thereby obtaining a title key. Theplayback device further decrypts the encrypted data using the title key,thereby obtaining, for example, a video stream and/or an audio stream.

The above mechanism has the following advantage. If a playback device,compositional elements, and a function (program) that have the risk ofbeing used in an unauthorized manner are already known when data istransmitted via the electronic distribution, the corresponding pieces ofidentification information are listed in the device list and aredistributed as part of the public key information. On the other hand,the playback device that has requested the distribution data inevitablyneeds to compare the pieces of identification information included inthe device list with the pieces of identification information of theplayback device, its compositional elements, and the like. As a result,if the playback device, its compositional elements, and the like areidentified in the device list, the playback device cannot use the publickey information for decrypting the encrypted data included in thedistribution data even if the combination of the identification numberof the SD memory card, the MKB, the encrypted title key, and the devicekey is correct. In this manner, it is possible to effectively preventdistribution data from being used in an unauthorized manner.

The identification information of the semiconductor memory card isdesirably recorded in a recording area having high confidentialityincluded in a recording area of the semiconductor memory card. This isbecause if the identification information such as the serial number ofthe SD memory card has been tampered with in an unauthorized manner, itis possible to realize an illegal copy of the SD memory card easily. Inother words, if the tampering allows generation of a plurality ofsemiconductor memory cards having the same identification information,it is impossible to distinguish between authorized products andunauthorized copy products by performing the above check (1). Therefore,it is necessary to record the identification information of thesemiconductor memory card on a recording area with high confidentialityin order to protect the identification information from being tamperedwith in an unauthorized manner.

The recording area with high confidentiality is structured within thesemiconductor memory card in the following manner, for example. First,as a recording area electrically disconnected from a recording area forrecording normal data (hereinafter, “first recording area”), anotherrecording area (hereinafter, “second recording area”) is provided. Next,a control circuit exclusively for accessing the second recording area isprovided within the semiconductor memory card. As a result, access tothe second recording area can be performed only via the control circuit.For example, assume that only encrypted data is recorded on the secondrecording area and a circuit for decrypting the encrypted data isincorporated only within the control circuit. As a result, access to thedata recorded on the second recording area can be performed only bycausing the control circuit to store therein an address of each piece ofdata recorded in the second recording area. Also, an address of eachpiece of data recorded on the second recording area may be stored onlyin the control circuit. In this case, only the control circuit canidentify an address of each piece of data recorded on the secondrecording area.

In the case where the identification information of the semiconductormemory card is recorded on the second recording area, then when anapplication program operating on the playback device acquires data fromthe distribution server via electronic distribution and records theacquired data in the semiconductor memory card, the following processingis performed. First, the application program issues an access request tothe control circuit via the memory card I/F for accessing theidentification information of the semiconductor memory card recorded onthe second recording area. In response to the access request, thecontrol circuit first reads the identification information from thesecond recording area. Then, the control circuit transmits theidentification information to the application program via the memorycard I/F. The application program transmits a transmission request ofthe distribution data together with the identification information. Theapplication program further records, in the first recording area of thesemiconductor memory card via the memory card I/F, the public keyinformation and the distribution data received from the distributionserver in response to the transmission request.

Note that it is preferable that the above-described application programcheck whether the application program itself has been tampered withbefore issuing the access request to the control circuit of thesemiconductor memory card. The check may be performed using a digitalcertificate compliant with the X.509 standard. Furthermore, it is onlynecessary to record the distribution data in the first recording area ofthe semiconductor memory card, as described above. Access to thedistribution data need not be controlled by the control circuit of thesemiconductor memory card.

<Application to Real-Time Recording>

The embodiment of the present invention is based on the assumption thatan AV stream file and a playlist file are recorded on a BD-ROM discusing the prerecording technique of the authoring system, and therecorded AV stream file and playlist file are provided to users.Alternatively, it may be possible to record, by performing real-timerecording, the AV stream file and the playlist file on a writablerecording medium such as a BD-RE disc, a BD-R disc, a hard disk, or asemiconductor memory card (hereinafter, “BD-RE disc or the like”) andprovide the user with the recorded AV stream file and playlist file. Insuch a case, the AV stream file may be a transport stream that has beenobtained as a result of real-time decoding of an analog input signalperformed by a recording device. Alternatively, the AV stream file maybe a transport stream obtained as a result of partializing a transportstream in digital form input by the recording device.

The recording device performing real-time recording includes a videoencoder, an audio encoder, a multiplexer, and a source packetizer. Thevideo encoder encodes a video signal to convert it into a video stream.The audio encoder encodes an audio signal to convert it into an audiostream. The multiplexer multiplexes the video stream and audio stream toconvert them into a digital stream in the MPEG-2 TS format. The sourcepacketizer converts TS packets in the digital stream in MPEG-2 TS formatinto source packets. The recording device stores each source packet inthe AV stream file and writes the AV stream file on the BD-RE disc orthe like.

In parallel with the processing of writing the AV stream file, thecontrol unit of the recording device generates a clip information fileand a playlist file in the memory and writes the files on the BD-RE discor the like. Specifically, when a user requests performance of recordingprocessing, the control unit first generates a clip information file inaccordance with an AV stream file and writes the file on the BD-RE discor the like. In such a case, each time a head of a GOP of a video streamis detected from a transport stream received from outside, or each timea GOP of a video stream is generated by the video encoder, the controlunit acquires a PTS of an I picture positioned at the head of the GOPand an SPN of the source packet in which the head of the GOP is stored.The control unit further stores a pair of the PTS and the SPN as oneentry point in an entry map of the clip information file. At this time,an “is_angle_change” flag is added to the entry point. Theis_angle_change flag is set to “on” when the head of the GOP is an IDRpicture, and “off” when the head of the GOP is not an IDR picture. Inthe clip information file, stream attribute information is further setin accordance with an attribute of a stream to be recorded. In thismanner, after writing the AV stream file and the clip information fileinto the BD-RE disc or the like, the control unit generates a playlistfile using the entry map in the clip information file, and writes thefile on the BD-RE disc or the like.

<Managed Copy>

The playback device according to the embodiments of the presentinvention may write a digital stream recorded on the BD-ROM disc 101 onanother recording medium via a managed copy. “Managed copy” refers to atechnique for permitting copy of a digital stream, a playlist file, aclip information file, and an application program from a read-onlyrecording medium such as a BD-ROM disc to a writable recording mediumonly in the case where authentication via communication with the serversucceeds. This writable recording medium may be a writable optical disc,such as a BD-R, BD-RE, DVD-R, DVD-RW, or DVD-RAM, a hard disk, or aportable semiconductor memory element such as an SD memory card, MemoryStick™, Compact Flash™, Smart Media™ or Multimedia Card™. A managed copyallows for limitation of the number of backups of data recorded on aread-only recording medium and for charging a fee for backups.

When a managed copy is performed from a BD-ROM disc to a BD-R disc or aBD-RE disc and the two discs have an equivalent recording capacity, thebit streams recorded on the original disc may be copied in order as theyare.

If a managed copy is performed between different types of recordingmedia, a trans code needs to be performed. This “trans code” refers toprocessing for adjusting a digital stream recorded on the original discto the application format of a recording medium that is the copydestination. For example, the trans code includes the process ofconverting an MPEG-2 TS format into an MPEG-2 program stream format andthe process of reducing a bit rate of each of a video stream and anaudio stream and re-encoding the video stream and the audio stream.During the trans code, an AV stream file, a clip information file, and aplaylist file need to be generated in the above-mentioned real-timerecording.

<Method for Describing Data Structure>

Among the data structures in the embodiments of the present invention, arepeated structure “there is a plurality of pieces of information havinga predetermined type” is defined by describing an initial value of acontrol variable and a cyclic condition in a “for” sentence. Also, adata structure “if a predetermined condition is satisfied, predeterminedinformation is defined” is defined by describing, in an “if” sentence,the condition and a variable to be set at the time when the condition issatisfied. In this manner, the data structure described in theembodiments is described using a high level programming language.Accordingly, the data structure is converted by a computer into acomputer readable code via the translation process performed by acompiler, which includes “syntax analysis”, “optimization”, “resourceallocation”, and “code generation”, and the data structure is thenrecorded on the recording medium. By being described in a high levelprogramming language, the data structure is treated as a part other thanthe method of the class structure in an object-oriented language,specifically, as an array type member variable of the class structure,and constitutes a part of the program. In other words, the datastructure is substantially equivalent to a program. Therefore, the datastructure needs to be protected as a computer related invention.

<Management of Playlist File and Clip Information File by PlaybackProgram>

When a playlist file and an AV stream file are recorded on a recordingmedium, a playback program is recorded on the recording medium in anexecutable format. The playback program makes the computer play back theAV stream file in accordance with the playlist file. The playbackprogram is loaded from a recording medium to a memory element of acomputer and is then executed by the computer. The loading processincludes compile processing or link processing. By these processes, theplayback program is divided into a plurality of sections in the memoryelement. The sections include a text section, a data section, a bsssection, and a stack section. The text section includes a code array ofthe playback program, an initial value, and non-rewritable data. Thedata section includes variables with initial values and rewritable data.In particular, the data section includes a file that is recorded on therecording medium and can be accessed at any time. The bss sectionincludes variables having no initial value. The data included in the bsssection is referenced in response to commands indicated by the code inthe text section. During the compile processing or link processing, anarea for the bss section is set aside in the computer's internal RAM.The stack section is a memory area temporarily set aside as necessary.During each of the processes by the playback program, local variablesare temporarily used. The stack section includes these local variables.When the program is executed, the variables in the bss section areinitially set at zero, and the necessary memory area is set aside in thestack section.

As described above, the playlist file and the clip information file arealready converted on the recording medium into computer readable code.Accordingly, at the time of execution of the playback program, thesefiles are each managed as “non-rewritable data” in the text section oras a “file accessed at any time” in the data section. In other words,the playlist file and the clip information file are each included as acompositional element of the playback program at the time of executionthereof. Therefore, the playlist file and the clip information filefulfill a greater role in the playback program than mere presentation ofdata.

INDUSTRIAL APPLICABILITY

The present invention relates to a technology for stereoscopic videoplayback and as described above, enables TS packets to be identifiedwith use of TS priority flag. It is therefore apparent that the presentinvention is industrially applicable.

REFERENCE SIGNS LIST

-   -   1500 VAU #1 in the dependent-view video stream    -   1510 PES packet    -   1512 PES payload    -   1520 TS packet sequence    -   1521 first group of the TS packet sequence 1520    -   1522 second group of the TS packet sequence 1520    -   1523 third group of the TS packet sequence 1520    -   1530 TS packet located at the end of the first group 1521    -   1531 TS header of the TS packet 1530    -   1532 AD field of the TS packet 1530    -   1533 TS payload of the TS packet 1530    -   1540 TS packet belonging to the second group 1522    -   1541 TS header of the TS packet 1540    -   1542 TS payload of the TS packet 1540    -   1550 TS packet located at the end of the second group 1522    -   1551 TS header of the TS packet 1550    -   1552 AD field of the TS packet 1550    -   1553 TS payload of the TS packet 1550    -   1560 TS packet belonging to the third group 1523    -   1561 TS header of the TS packet 1560    -   1562 TS payload of the TS packet 1560

1-4. (canceled)
 5. A semiconductor integrator circuit for receiving aseries of data including a main-view video stream, a sub-view videostream, and a graphics stream, and then performing video signalprocessing on the received data, wherein the main-view video stream usedto provide a plurality of main-views of a plurality of stereoscopicvideo images, a decoded main-view video stream being presented on amain-view video plane; the sub-view video stream used to provide aplurality of sub-views of the plurality of stereoscopic video images, adecoded sub-view video stream being presented on a sub-view video plane;and the graphics stream used to provide a plurality of graphics images,a decoded graphics stream being presented on a graphics plane, whereinthe sub-view video stream is multiplexed in a transport stream, aplurality of transport packets being formed from the transport stream,wherein each of the plurality of transport packets has informationindicating whether the transport packet contains offset information,wherein the sub-view video stream includes the offset information,wherein a first shifted graphics plane and a second shifted graphicsplane are formed based on the graphics plane on which the decodedgraphics stream is presented and the offset information, wherein dataformed from the main-view video plane on which the decoded main-viewvideo stream is presented and data formed from the first shiftedgraphics plane are overlaid, and wherein data formed from the sub-viewvideo plane on which the decoded sub-view video stream is presented anddata formed from the second shifted graphics plane are overlaid, thesemiconductor integrated circuit comprising: a main control unitoperable to control the semiconductor integrated circuit; a streamprocessing unit operable to receive the series of data, temporarilystore the series of data into a memory internal or external to thesemiconductor integrated circuit, and then demultiplex the series ofdata into the picture data and the graphics data; a signal processingunit operable to decode the picture data and the graphics data; and anAV output unit operable to output the picture data decoded by the signalprocessing unit, wherein the stream processing unit comprises aswitching unit operable to receive the series of data and switch storagelocation thereof between a first area and a second area in the memory,the main control unit controls the switching unit to store picture databelonging to the main-view data groups and the sub-view data groups inthe first area and the second area, respectively, among the picture datadecoded, data belonging to the main-view data groups is stored into athird area in the memory, the third area corresponding to a main-viewvideo plane, among the picture data decoded, data belonging to thesub-view data groups is stored into a fourth area in the memory, thefourth area corresponding to a sub-view video plane, the graphics datadecoded is stored into a fifth area in the memory, the fifth areacorresponding to a graphics plane, the AV output unit comprises an imagesuperimposing unit operable to superimpose one of the graphics datadecoded and the picture data decoded on the other, the signal processingunit monitors TS priority flags of TS packets included in the series ofdata, and extracts the metadata, the image superimposing unit provides apositive offset and a negative offset for horizontal coordinates in thegraphics plane to generate a pair of graphics planes according to offsetinformation included in the metadata extracted by the signal processingunit, and then superimposes the pair of graphics planes separately onthe main-view video plane and the sub-view video plane, and the AVoutput unit outputs the picture data decoded after the graphics datadecoded is superimposed thereon.